Reka Flash 3 - An open-source inference model from Reka AI

What is Reka Flash 3?

Reka Flash 3 is an open-source inference model from Reka AI, boasting 2.1 billion parameters. It supports multimodal input, including text, images, video, and audio, and can handle context lengths of up to 32k tokens. Suitable for scenarios such as general dialogue, coding assistance, and instruction following, the model's full-precision size is 39GB (fp16), which can be compressed to 11GB after 4-bit quantization, facilitating deployment on devices.

Main functions of Reka Flash 3

• Multimodal input supportReka Flash 3 can handle various types of input data, including text, images, video, and audio.
• High-efficiency reasoning abilityReka Flash 3 supports the most 32k tokens The context length allows it to handle longer documents and complex tasks, such as long papers, code snippets, or complex multi-step problems.
• Multilingual supportReka Flash 3 performs well in multilingual support, primarily focusing on English, achieving a COMET score of 83.2 at WMT’23, thus meeting the needs of users with different languages.

The technical principles of Reka Flash 3

• Modular encoder-decoder architectureReka Flash 3 employs a modular encoder-decoder architecture. The encoder handles various input data types, including text, images, video, and audio, converting them into high-dimensional vector representations. The decoder then generates text output based on the encoder's output.
• Reinforcement learning trainingReka Flash 3's training combines publicly available and synthetic datasets, employing the REINFORCE Leave One-Out (RLOO) method for reinforcement learning. This achieves a balance between capability and efficiency, enabling the model to better understand and generate high-quality content.
• Budgetary enforcement mechanism: Through specific <reasoning> Labels allow users to limit the inference steps of a model, ensuring that the model maintains efficient and consistent performance with limited computing resources.

Reka Flash 3 project address

• HuggingFace model library:https://huggingface.co/RekaAI/reka-flash-3

Application scenarios of Reka Flash 3

• General conversationReka Flash 3 enables natural and fluent conversations and provides services such as companionship and information retrieval.
• Coding aidThe model can help developers write code, debug code problems, and provide code optimization suggestions.
• Instruction FollowReka Flash 3 can understand and execute complex instructions to complete various tasks, such as generating reports and analyzing data.
• Internet search assistanceReka Flash 3 can utilize web search functionality to retrieve additional information to assist in answering questions.