AB
AB
AiBoss

All Categories

project

OmniSVG - Fudan University and StepFun jointly launch an end-to-end multimodal vector graphics generation model

OmniSVG is the world's first end-to-end multimodal SVG (Scalable Vector Graphics) generation model, jointly developed by Fudan University and StepFun. Based on a pre-trained Visual Language Model (VLM), it utilizes an innovative SVG tokenization method...

What is OmniSVG?

OmniSVG is the world's first end-to-end multimodal SVG (Scalable Vector Graphics) generation model, jointly developed by Fudan University and StepFun. Based on a pre-trained Visual Language Model (VLM), it uses an innovative SVG tokenization method to parameterize SVG commands and coordinates as discrete tokens, achieving decoupling between structural logic and geometric details. This enables OmniSVG to efficiently generate a diverse range of high-quality SVG graphics, from simple icons to complex anime characters.

Main functions of OmniSVG

  • Multimodal generationOmniSVG is the first end-to-end multimodal SVG generation model that can generate high-quality SVG graphics based on text descriptions, image references, or character references. It can generate a wide variety of graphics, from simple icons to complex anime characters.
  • Efficient generation and trainingBased on the pre-trained Visual Language Model (VLM) Qwen-VL, OmniSVG employs an innovative SVG tokenization method to parameterize SVG commands and coordinates as discrete tokens, separating structural logic from geometric details during training. This improves training efficiency by more than 3 times compared to traditional methods, enabling the processing of sequences of up to 30,000 tokens and supporting the generation of complex SVGs with rich details.
  • Datasets and EvaluationThe OmniSVG team released the MMSVG-2M dataset, containing 2 million SVG resources with multimodal annotations, covering three subsets: icons, illustrations, and characters. They also proposed a standardized evaluation protocol, MMSVG-Bench, for testing the performance of conditional SVG generation tasks.
  • Editability and UsabilityThe generated SVG files are infinitely scalable and fully editable, and can be seamlessly integrated into professional design workflows, such as tools like Adobe Illustrator, improving the practicality of AI-generated graphics in graphic design, web development, and other fields.

OmniSVG's technical principles

  • Based on pre-trained visual language model (VLM)OmniSVG is built upon the pre-trained visual-language model Qwen-VL. This model deeply fuses image and text information, providing a powerful foundation for multimodal generation.
  • SVG tokenization methodOmniSVG innovatively parameterizes SVG commands and coordinates as discrete tokens, processing SVG generation in a manner similar to natural language processing. This improves training efficiency while retaining the ability to generate complex SVG structures.
  • End-to-end multimodal generation frameworkOmniSVG supports the direct generation of SVG graphics from various input methods, including text descriptions, image references, and character references. This end-to-end generation framework produces richly colored and vividly detailed vector graphics, overcoming many limitations of traditional methods.
  • Efficient training and long sequence processingCompared to traditional methods, OmniSVG offers more than a 3x speedup in training and can handle sequences of up to 30,000 tokens. This enables it to generate complex SVG graphics with rich detail.

OmniSVG project address

Application scenarios of OmniSVG

  • Brand icon designOmniSVG can quickly generate brand icons based on text descriptions, eliminating the need for designers to draw them from scratch and significantly reducing manual design time.
  • Web developmentIcons are an indispensable element in web development. OmniSVG can generate vector icons from text descriptions or image references. These icons can be scaled without loss of quality and are suitable for a variety of resolutions, from mobile devices to 4K displays.
  • Character and Scene DesignIn game development, OmniSVG can be used to generate graphic assets such as game characters and scenes, adding a unique artistic style to the game.
  • Dynamic character generationBased on character references, OmniSVG can generate vector graphics that retain the same character features but differ in pose or scene.
  • Rapid prototypingContent creators can use OmniSVG to quickly generate prototypes of icons, illustrations, or character graphics, accelerating the creation process.
Back