GLM-4.1V-Thinking - A series of open-source visual language models from Zhipu AI.

What is GLM-4.1V-Thinking?

GLM-4.1V-Thinking is an open-source visual language model launched by Zhipu AI, designed specifically for complex cognitive tasks, and supports multimodal inputs such as images, videos, and documents. Based on the GLM-4V architecture, the model introduces a thought chain reasoning mechanism and utilizes a curriculum sampling reinforcement learning strategy to systematically improve cross-modal causal reasoning capabilities and stability. The lightweight version, GLM-4.1V-9B-Thinking (GLM-4.1V-9B-Base and GLM-4.1V-9B-Thinking both possess deep thinking and reasoning capabilities), has a parameter count controlled at the 10B level. In 28 authoritative evaluations, it achieved the best performance among 10B-level models in 23 of them, with 18 of these performances matching or surpassing the 72B parameter model Qwen-2.5-VL, demonstrating the ultimate performance potential of a small-volume model.

Main functions of GLM-4.1V-Thinking

• Image understandingIt accurately identifies and analyzes image content, supporting complex visual tasks such as object detection, image classification, and visual question answering.
• Video processingIt has the ability to perform time-series analysis and event logic modeling, and supports processing video input to perform video understanding, video description, and video question answering.
• Document parsingIt supports processing images and text content in documents, and supports long document comprehension, chart comprehension, and document Q&A.
• Mathematical and Scientific ReasoningIt supports complex mathematical problem solving, multi-step deduction and formula understanding, and can handle reasoning tasks in the STEM field.
• Logical reasoningIt supports logical reasoning and causal analysis, and supports complex reasoning tasks such as multi-step reasoning and logical judgment.
• Cross-modal reasoningIt combines visual and linguistic information for reasoning, supporting tasks such as image and text understanding, visual question answering, and visual anchoring.

Technical Principles of GLM-4.1V-Thinking

• Architecture DesignThe system uses AIMv2Huge as the visual encoder to process and encode image and video inputs. An MLP adapter aligns visual features to the token space of the language model. The language decoder uses a GLM as the language model to process multimodal tokens and generate the output.
• Training methodsWe build a robust visual language foundation model by pre-training on large-scale image-text pairs, academic literature, and knowledge-intensive data. Supervised fine-tuning using Long Chain Inference (CoT) data enhances the model's reasoning ability and improves its alignment with human understanding. Curriculum Sampling Reinforcement Learning (RLCS) dynamically selects the most informative samples for training, improving the model's performance across various tasks.
• Technological innovationThis system introduces a thought chain reasoning mechanism, allowing the model to gradually generate detailed reasoning processes. Based on a course sampling strategy, the difficulty of training samples is dynamically adjusted to ensure the model receives the most effective training at different stages. Utilizing 2D-RoPE and 3D-RoPE technologies, it supports image input with arbitrary resolution and aspect ratios, enhancing the model's spatiotemporal understanding capabilities.

Performance of GLM-4.1V-Thinking

In 23 out of 28 authoritative evaluations, including MMStar, MMMU-Pro, ChartQAPro, and OSWorld, the model achieved the best results among 10B-level models, with 18 of them matching or surpassing the Qwen-2.5-VL model with a parameter count as high as 72B.

Project address for GLM-4.1V-Thinking

• GitHub repositoryhttps://github.com/THUDM/GLM-4.1V-Thinking
• HuggingFace model library: https://huggingface.co/collections/THUDM/glm-41v-thinking-6862bbfc44593a8601c2578d
• arXiv technical paperhttps://arxiv.org/pdf/2507.01006v1
• Experience the demo online:https://huggingface.co/spaces/THUDM/GLM-4.1V-9B-Thinking-Demo

How to use GLM-4.1V-Thinking

• API Interface:
  • Register an accountVisit the Zhipu AI Open Platform, register an account and log in.
  • Get API KeyCreate an application on the platform and obtain a unique API Key.
  • Calling the APIAccording to the API documentation, use an HTTP request to call the model interface, send input data, and obtain the model's output results.For example, here's a code example of using Python to call an API:

import requests
import json

# 设置API接口地址和API Key
api_url =
"https://api.zhipuopen.com/v1/glm-4.1v-thinking"
api_key =
"your_api_key"
# 准备输入数据
input_data =
{
"image":
"image_url_or_base64_encoded_data",
"text":
"your_input_text"
}
# 设置请求头
headers =
{
"Authorization":
f"Bearer {api_key}",
"Content-Type":
"application/json"
}
# 发送请求
response = requests.post(api_url, headers=headers, data=json.dumps(input_data))
# 获取结果
result = response.json()
print(result)

• Open source model:
  • Download ModelVisit Hugging Face, find the GLM-4.1V-Thinking model page, and download the model file.
  • Loading Model: Load the model using the appropriate deep learning framework (such as PyTorch).
  • To reasonThis involves preprocessing the input data before feeding it into the model and then obtaining the model's output. For example, here's a code example of loading a model and performing inference using PyTorch:

from transformers import AutoModelForVision2Seq, AutoProcessor
import torch

# 加载模型和处理器
model_name =
"THUDM/glm-4.1v-thinking"
model = AutoModelForVision2Seq.from_pretrained(model_name)
processor = AutoProcessor.from_pretrained(model_name)
# 准备输入数据
image_url =
"image_url_or_image_path"
text =
"your_input_text"
inputs = processor(images=image_url, text=text, return_tensors="pt")
# 进行推理
with torch.no_grad():
 outputs = model(**inputs)
# 获取结果
result = processor.decode(outputs.logits[0], skip_special_tokens=True)
print(result)

• Online experience platform:
  • Access Experience LinkVisit the GLM-4.1V-Thinking experience page on the Hugging Face platform.
  • Input dataUpload images or enter text on a webpage.
  • Get ResultsClick the "Run" button, wait for the model to process, and view the results.

Application scenarios of GLM-4.1V-Thinking

• Educational guidanceIt assists students in solving complex problems in subjects such as mathematics and science, providing detailed solution steps and reasoning processes to help students better understand and master knowledge.
• Content creationIt combines images and text to generate creative content, such as advertising copy, social media posts, and news reports, improving the efficiency and quality of content creation.
• Intelligent InteractionAs an intelligent customer service representative or virtual assistant, it understands users' questions and needs, provides accurate and timely answers and solutions, and supports multimodal input.
• Industry ApplicationsIn fields such as healthcare, finance, and industry, it assists professionals in tasks such as data analysis, report generation, and equipment monitoring, improving work efficiency and accuracy.
• Entertainment and LifeIt provides travel guides and attraction introductions, recommends dishes and cooking methods for food, and generates storylines and mission designs for games, enriching users' entertainment experience.