Spatial-TTT - A streaming visual spatial intelligence framework jointly developed by Tsinghua University and Hunyuan Open Source.

What is Spatial-TTT?

Spatial-TTT is a streaming visual spatial intelligence framework jointly developed by Tsinghua University, Tencent Hunyuan, and Nanyang Technological University. With only 2B parameters, the framework continuously updates its internal spatial memory during video streaming using Test-Time Training (TTT) technology, enabling spatial inference on videos up to 120 minutes long. On multiple benchmarks such as VSI-Bench and MindCube-Tiny, it outperforms closed-source large models like GPT-5 and Gemini-3-pro, achieving peak GPU memory usage that is over 40% lower than industry-leading solutions.

Main functions of Spatial-TTT

• Streaming Space Memory MaintenanceThe video stream is input in chunks, and the model parameters (fast weights) are updated online to continuously accumulate 3D spatial evidence without loading the entire video at once.
• Long-term spatial reasoningIt supports tasks such as relative/absolute distance estimation, object counting and size determination, room size perception, direction recognition, path planning and appearance order reasoning.
• Ultra-long video processingIt can stably handle continuous video streams from 10 to 120 minutes, maintaining performance without degradation on the VSI-SUPER long-term memory benchmark.
• Efficient ReasoningWith a peak video memory of only 11.9GB at 1024 frames per second input, the theoretical computational cost is 799.4 TFLOPs, which is significantly lower than similar large models.

Follow us on WeChat and reply with "open source",join inAI open source project discussion group

Spatial-TTT Technical Principles

• Hybrid TTT architectureThe research team interleaved TTT layers and standard self-attention anchoring layers in the decoder at a ratio of 3:1. 75% of the TTT layers are responsible for writing long-range spatial information into fast weights, achieving memory expansion with linear complexity; 25% of the full attention layers retain the semantic understanding and cross-modal alignment capabilities of the pre-trained model, avoiding the destruction of the original visual-language capabilities by the pure TTT structure.
• Spatial prediction mechanismTraditional TTT uses point-by-point linear projection to generate Q.
  /K/V ignores the local geometry of visual tokens. Spatial-TTT introduces lightweight 3D spatiotemporal convolution in the TTT branch, enabling fast weight learning of spatiotemporal context-to-spatiotemporal context prediction relationships, mappings of non-isolated tokens, and stably capturing geometric correspondences, viewpoint changes, and temporal continuity.
• Dense Scene Description Supervision and Progressive TrainingExisting spatial intelligence data is mostly sparse and localized, making it difficult to drive rapid weight learning and global updates. The research team constructed dense 3D scene description data covering global context, object categories and quantities, and spatial relationships, and adopted a two-stage progressive training of spatial perception: the first stage learns to "remember the entire space" through dense description, forming a global 3D awareness; the second stage uses millions of spatial VQA data to enhance streaming reasoning ability.

How to use Spatial-TTT

• Environmental preparationAccess the GitHub repository (https://github.com/THU-SI/Spatial-TTT/) to clone the code and configure the Python environment and dependencies.
• Model loadingLoad the pre-trained Spatial-TTT-2B weights and initialize the TTT fast weights.
• Input video streamThe long video is divided into continuous chunks, which are then input into the model one chunk at a time. After each chunk is processed, the model automatically updates its internal spatial state.
• Space Q&AInput a natural language spatial question (such as "Starting from the red candle and facing the window, how do you get to the door?"), and the model generates an answer based on accumulated spatial memory.
• Extended ApplicationsIt can connect to real-time video streams from robots, autonomous driving systems, or AR devices to achieve continuous spatial perception.

Spatial-TTT's core advantages

• Small parameters outperform large modelsThe 2B parameter outperforms closed-source giants such as GPT-5 and Gemini-3-pro on multiple spatial benchmarks, and MindCube-Tiny's accuracy is 12 percentage points higher than Gemini-3-pro.
• Linear scalabilityBy updating weights online via fast weights, the amount of video memory and computation increases linearly with the video length, avoiding the quadratic complexity explosion of traditional long context models.
• Spatial memory rather than content accumulationInstead of relying on an infinitely expanding context window, it transforms observations into an internal space state that can be updated, corrected, and invoked, making it more suitable for long-term operation of physical agents.
• Significant improvement in video memory efficiencyAt 1024 frames per second, the video memory usage is 11.9GB, which is more than 40% lower than the industry-leading solution (21.2GB). Furthermore, the comparison model with an explicit geometry encoder cannot run at the same frame rate.

Spatial-TTT project address

• Project official websitehttps://liuff19.github.io/Spatial-TTT/
• GitHub repositoryhttps://github.com/THU-SI/Spatial-TTT
• HuggingFace model libraryhttps://huggingface.co/collections/THU-SI/spatial-ttt
• arXiv technical paperhttps://arxiv.org/pdf/2603.12255

Spatial-TTT's Competitive Product Comparison

Application scenarios of Spatial-TTT

• Robot NavigationAs service robots or home robots move continuously in a room, they update the spatial map in real time, answering questions such as "the position of the sofa relative to the window" or planning a path to reach the target point.
• autonomous drivingThe vehicle continuously accumulates spatial relationships between roads, intersections and obstacles during long-distance driving, supporting complex direction judgments and distance estimations.
• AR/VR Spatial InteractionThe head-mounted display continuously understands the environment layout during user operation, enabling spatial anchoring and object relationship queries that blend the virtual and real worlds.
• Intelligent monitoring and securityIt performs online spatial analysis on long-duration video streams, tracking the order of object appearance, counting, and positional changes without requiring offline full-scale processing.