MARS - ByteDance launches a framework to optimize the training efficiency of large models.
MARS (Make vAriance Reduction Shine) is an innovative optimization framework launched by ByteDance to improve the efficiency of training large models. MARS integrates preconditioning gradient methods with variance reduction techniques, based on scaled stochastic recursion...
What is MARS?
MARS (Make vAriance Reduction Shine) is an innovative optimization framework launched by ByteDance to improve the efficiency of training large-scale models. MARS integrates preconditioning gradient methods with variance reduction techniques, optimizing gradient estimation based on scaling stochastic recursive momentum. The MARS framework is flexible, supporting full-matrix or diagonal Hessian approximations, and derives three optimization algorithm examples based on AdamW, Lion, and Shampoo. Experimental results show that MARS exhibits superior performance compared to the traditional AdamW optimizer when training GPT-2 models.
Main functions of MARS
- Improve training efficiencyMARS combines preconditioning gradient methods and variance reduction techniques to improve the efficiency of training large models, especially when training deep neural networks and large language models.
- Unified optimization frameworkIt provides a unified framework that adapts to different Hessian approximation methods, including full matrix and diagonal matrix approximations.
- Algorithm instantiationWithin the MARS framework, three specific optimization algorithm examples are implemented: MARS-AdamW, MARS-Lion, and MARS-Shampoo, each based on a different preconditional gradient update strategy.
- Variance reductionThe introduction of scaling stochastic recursive momentum effectively reduces gradient variance during training and accelerates model convergence.
MARS Technical Principles
- Preconditional gradient methodThe learning rate is adjusted based on the preconditioning gradient method, so that each parameter or group of parameters has a customized learning rate to adapt to its local curvature.
- Variance reduction techniquesIntroducing variance reduction techniques, such as STORM (Stochastic Recursive Momentum), reduces the variance of stochastic gradients and accelerates the optimization process.
- Scaling random recursive momentumBased on STORM, a scaling parameter is introduced to adjust the strength of variance reduction and a new gradient estimator is defined.
- Gradient clipping and exponential moving averageTo optimize training stability, MARS applies gradient clipping in the gradient estimator and calculates recursive momentum using exponential moving average (EMA).
MARS project address
- arXiv technical paper:https://arxiv.org/pdf/2411.10438
Application scenarios of MARS
- Deep learning model trainingTraining deep neural networks, especially complex models with many parameters.
- Large-scale language modelsOptimize the training process of large language models, such as the GPT series models, to improve training efficiency and model performance.
- Computer vision tasksIn computer vision fields such as image classification and object detection, it accelerates model training and improves the model's generalization ability.
- Reinforcement learning algorithmsIn reinforcement learning, the parameters of the policy network or value function are optimized, especially when dealing with high variance gradients.
- Recommender system modelWhen building a recommendation system, optimize model parameters to better handle the characteristics of large-scale users and items.