LingBot-Map - An open-source streaming 3D reconstruction model from LingBot.

LingBotWhat is a Map?

LingBot-Map is an open-source streaming 3D reconstruction model from Antminer. It enables real-time camera pose estimation and scene 3D structure reconstruction using ordinary RGB cameras during video capture. The model is based on pure autoregressive modeling and a geometric context Transformer architecture. On authoritative benchmarks such as Oxford Spires, it achieves approximately 2.8 times higher trajectory accuracy than previous best streaming methods, filling a key technological gap in the field of real-time spatial perception.

LingBot-Main functions of Map

• Real-time streaming reconstructionIt can understand while watching, process the current and historical images frame by frame, and continuously output camera pose and depth information.
• Stable operation of long sequencesSupports continuous inference of 10,000+ long video frames with almost no accuracy degradation during long sequence execution.
• Pure visual spatial perceptionNo complex hardware is required; a single ordinary camera can achieve real-time 3D mapping.
• Camera trajectory estimationSupports accurate estimation of the camera's motion trajectory in three-dimensional space (pose estimation).

How to use LingBot-Map

• Environmental preparationEnsure your local environment has an NVIDIA GPU (recommended VRAM ≥ 12GB), and that Python 3.8+, PyTorch 2.0+, and the CUDA toolchain are installed. Your system must support Linux or Windows WSL2 to ensure compatibility with dependency library compilation.
• Installation and DeploymentAccess the GitHub repository https://github.com/Robbyant/lingbot-map After cloning the code and navigating to the project directory, install the dependencies using pip:pip install -r requirements.txtThe environment will automatically install the DINO backbone, Transformer architecture, and Open3D libraries required for 3D visualization.
• Model AcquisitionFrom HuggingFace (robbyant/lingbot-mapAlternatively, you can download the pre-trained weights from ModelScope and place them in your project. checkpoints/ Table of Contents. The model includes the Geometric Context Attention (GCA) core network and the Camera/Depth prediction head.
• Running inference:
  • Offline video modeThe input is a monocular RGB video file. The model extracts DINO features frame by frame, processes them through Frame Attention and GCA layers, and outputs camera pose trajectory and depth map. Finally, they are fused into a 3D point cloud map.
  • Live camera modeConnect a regular USB camera, set the input stream resolution to 640×480 or 1280×720, use ~20 FPS for real-time inference of the model, continuously output the current camera pose and incrementally update the 3D structure of the scene.
• Output ResultsThe reconstruction results include camera trajectory files (standard coordinate format) and scaled dense point clouds, and the 3D reconstruction effect can be viewed through a visualization script. During long sequence inference, the GCA mechanism automatically manages memory, supporting continuous processing of 10,000+ frames without restarting.
• Advanced configuration:Adjustment config.yaml In anchor_context and trajectory_memory The parameters can balance accuracy and computational cost; for large-scale scenarios, local window optimization (Local Pose-Reference Window) can be enabled to improve long-trajectory stability. For specific API call examples and parameter descriptions, please refer to the GitHub repository. README.md and demo.py.

LingBot-Key information and usage requirements of the Map

• Development TeamRobbyant Technology
• Open source licenseThe model weights and code are open source (GitHub, HuggingFace, ModelScope).
• Hardware RequirementsRequires GPU support (recommended for environments with sufficient video memory)
• Reasoning speedApproximately 20 FPS real-time inference

LingBot-Map's core advantages

• Leading in precisionThe absolute trajectory error (ATE) on the Oxford Spires dataset is only 6.42 meters, which is significantly better than the offline methods DA3 (12.87 meters) and VIPE (10.52 meters); the F1 score of the ETH3D benchmark reconstruction reaches 85.70, which is more than 8% better than the second place.
• Real-time and efficient20 FPS stably supports robot operations, and the computational and storage overhead is almost constant under long sequences.
• Low hardware thresholdNo depth camera or LiDAR is required; a regular RGB camera can operate.
• End-to-end learningBreaking away from the limitations of traditional SLAM, which relies on manual design and complex optimization, the core logic is learned uniformly by the model.

LingBot-Map's project address

• Project official websitehttps://technology.robbyant.com/lingbot-map
• GitHub repositoryhttps://github.com/Robbyant/lingbot-map
• HuggingFace model libraryhttps://huggingface.co/robbyant/lingbot-map
• arXiv technical paper: https://arxiv.org/pdf/2604.14141

LingBot-Comparison of Map's similar competing products

LingBot-Application Scenarios of Map

• Robot navigation and obstacle avoidanceIt provides real-time spatial awareness capabilities for mobile robots and supports continuous online processing.
• autonomous drivingIt can build real-time 3D maps of the environment to help vehicles understand the physical spatial structure.
• Embodied Intelligence: Serves as the spatial perception base for the LingBot series (including Depth, VLA, World, VA and other models), supporting robot operation and interaction.
• AR/VR Spatial ComputingReal-time reconstruction of the physical environment enables precise alignment between the virtual and real worlds.