FlexTok - An image processing technology jointly developed by Apple and EPFL

What is FlexTok?

FlexTok is an image processing technology jointly developed by the Swiss Federal Institute of Technology in Lausanne (EPFL) and Apple. It achieves efficient image compression and generation by resampling two-dimensional images into one-dimensional discrete token sequences, describing images with flexible lengths. FlexTok's core technologies include dynamic pixel reassembly, which can improve image compression rates by 300%, support real-time rendering of 8K video, and significantly reduce power consumption.

FlexTok's main functions

• High-efficiency image compressionThrough dynamic pixel recombination technology, FlexTok can flexibly adjust the number of markers according to the complexity of the image, improving the image compression rate by 300%, while also supporting real-time rendering of 8K video.
• Low power consumption and high performanceWhen processing high-resolution images, FlexTok reduces power consumption by 45%, significantly improving the device's energy efficiency.
• Lossless super-resolution reconstructionFlexTok is the first to achieve lossless super-resolution reconstruction on mobile devices, enabling high-quality upscaling of low-resolution images.
• Flexible image generationWith its "visual vocabulary," FlexTok can describe images from coarse to fine detail, supporting high-fidelity image generation and image generation under textual conditions.

The technical principles of FlexTok

• Dynamic pixel recombination technologyFlexTok rearranges and compresses the pixel information of an image into discrete token sequences through dynamic pixel recombination.
• Multi-scale discretization processingFlexTok borrows the idea of a multi-scale quantization autoencoder (VQ-VAE) to progressively decompose an image from high resolution into a sequence of low-resolution discrete markers. The generation process proceeds gradually from coarse to fine, similar to the hierarchical processing of human vision.
• Application of autoregressive modelsFlexTok uses an autoregressive model to model discrete token sequences. The autoregressive model generates images by progressively predicting the next token, similar to how a language model generates text. It can capture local structural and detailed information of images, achieving high-quality image generation.

FlexTok project address

• Project official website:https://flextok.epfl.ch/
• arXiv technical paper:https://arxiv.org/pdf/2502.13967

Application scenarios of FlexTok

• Image processing for smart home devicesFlexTok's efficient compression technology can be used in image sensors in smart home devices, such as smart cameras or smart locks. By optimizing the transmission and storage of image data, it can reduce storage space usage and network bandwidth consumption without compromising image quality.
• Image optimization in home entertainment systemsIn home theaters or smart TVs, FlexTok's super-resolution reconstruction capabilities can be used to improve the image quality of low-resolution videos, maintaining a clear visual effect on a large screen.
• Intelligent security monitoringFor home security cameras, FlexTok's technology enables more efficient image compression and storage, while also improving the clarity of the monitoring image through super-resolution technology, helping users to more accurately identify details in the image.
• Image management in mobile devicesOn smartphones or tablets, FlexTok helps users store and manage large numbers of photos more efficiently, while improving the display quality of photos through lossless super-resolution technology.