DistriFusion - A distributed parallel inference framework for high-resolution diffusion models

What is DistriFusion?

DistriFusion is a distributed parallel inference framework for high-resolution diffusion models, significantly accelerating the generation of high-resolution images across multiple GPUs through distributed parallel inference. DistriFusion utilizes patch parallelism to segment and distribute images across different devices, achieving up to 6x faster inference speeds without additional training while maintaining image quality. The implementation of DistriFusion provides a new benchmark for AI content creation and parallel computing research.

Main functions of DistriFusion

• Distributed Parallel InferenceThe inference process of the diffusion model is executed in parallel on multiple GPUs to improve the speed of image generation.
• Image segmentationHigh-resolution images are divided into multiple patches, and each patch is processed independently to achieve parallelization.
• No additional training requiredAs a training-free algorithm, DistriFusion can be directly applied to existing diffusion models, such as Stable Diffusion XL.
• Maintain image qualityWhile accelerating image generation, optimization techniques are used to maintain the high quality of the generated images.
• Asynchronous communicationIt supports asynchronous data exchange, reducing latency caused by communication overhead.

DistriFusion's technical principles

• Patch ParallelismDistriFusion divides the input image into multiple patches, each of which can be processed independently on a different GPU, thus achieving parallelization.
• Asynchronous communicationDuring parallel processing, DistriFusion employs an asynchronous communication mechanism, enabling data exchange between GPUs without blocking the computation process, thereby reducing waiting time and improving efficiency.
• Utilizing the sequential nature of the diffusion processDistriFusion observed that the inputs of adjacent steps in the diffusion model are highly similar, and therefore proposed a method to provide contextual information for the current step by reusing the feature map of the previous time step.
• Shifted Patch ParallelismThis simulates interactions between patches by making small displacements of the patches at each time step, without requiring explicit global communication.
• Pipeline computingDistriFusion's design allows for the pipelined computation process, enabling different GPUs to work simultaneously at different time steps, further improving processing speed.
• No need to sacrifice image qualityDistriFusion significantly accelerates the image generation process, ensuring that the quality of the generated images is not compromised through a carefully designed algorithm.
• Applicable to multiple diffusion modelsDistriFusion is not limited to a specific diffusion model; it can be applied to a variety of existing diffusion models, such as Stable Diffusion XL, to achieve acceleration.

DistriFusion's project address

Application scenarios of DistriFusion

• AI Art CreationIn the arts, DistriFusion can quickly generate high-quality images, helping artists and designers realize their creative ideas.
• Game and film productionIn the production of visual effects for games and movies, DistriFusion can accelerate the rendering process and shorten the production cycle.
• Virtual Reality (VR) and Augmented Reality (AR)In VR and AR applications, DistriFusion can quickly generate realistic 3D environments and scenes.
• Data visualizationIn the field of data analysis, DistriFusion can be used to generate complex visualizations to help users understand data more intuitively.
• Advertising and MarketingDistriFusion can be used to quickly generate attractive advertising images and marketing materials, improving the appeal and effectiveness of advertising.