Xiaohongshu has open-sourced BigMac, a multimodal, large-scale model pipelined parallel training framework.

The Dots Infra team at Xiaohongshu has open-sourced BigMac, a multimodal large-scale model training framework. Addressing the challenge of balancing computational efficiency and memory usage, BigMac proposes a secure nested pipeline design, using an LLM pipeline as the backbone to sequentially embed encoder and generator computations. Experiments show that its training speed is 1.08 to 1.9 times faster than the baseline, while maintaining stable memory usage. It is currently being used for production training of the Dots series of models.