EXAONE 3.5 - An open-source AI model released by LG, adept at handling long texts and reducing model illusion problems.
EXAONE 3.5 is an open-source AI model released by LG AI Research Institute, comprising three versions with 2.4 billion, 7.8 billion, and 32 billion parameters. EXAONE 3.5 excels in long text processing, demonstrating superior performance in benchmark tests, particularly in practical applications and long text processing...
What is EXAONE 3.5?
EXAONE 3.5 is an open-source AI model from LG AI Research Institute, available in three versions with 2.4 billion, 7.8 billion, and 32 billion parameters. EXAONE 3.5 excels in long text processing, demonstrating superior performance in benchmark tests, particularly in practical applications, long text processing, and mathematics. The model employs retrieval-enhanced generation technology and multi-step inference capabilities to effectively reduce error messages and improve accuracy. LG plans to further expand its AI capabilities and launch ChatEXAONE, an enterprise-grade AI agent service with sophisticated query analysis and user-defined search functions, equipped with encryption and privacy protection technologies to ensure secure use within the company.
Main features of EXAONE 3.5
- Multi-version model supportIt provides three models with different parameter scales to adapt to different application scenarios and computing resource constraints.
- Instruction compliance capabilityIt demonstrates excellent command compliance in real-world scenarios and achieved top scores in multiple benchmark tests.
- Long context understandingIt excels in long text processing, effectively understanding and handling contexts of up to 32K tokens.
- Bilingual abilityExcellent bilingual skills in Korean and English, especially outstanding performance in benchmark tests in both Korean and English.
- Search Enhancement Generation Technology: Use search-enhanced generation techniques to generate answers based on reference documents or web search results.
- Multi-step reasoning abilityIt possesses multi-step reasoning ability, effectively reducing the "hallucination" phenomenon and improving the accuracy of answers.
The technical principles of EXAONE 3.5
- Transformer architectureIt is a deep learning model based on the latest decoder-only Transformer architecture, used for processing sequential data.
- Long context processingThe maximum context length has been increased from 4,096 tokens in EXAONE 3.0 to 32,768 tokens by using long context fine-tuning technology.
- Pre-training and post-training:
- Pre-trainingThe first stage involves pre-training with a large training corpus, while the second stage involves data collection and pre-training for areas that need improvement, particularly enhancing the ability to understand long contexts.
- Post-trainingThis includes supervised fine-tuning (SFT) and preference optimization, which enhance the model's ability to follow instructions and its consistency with human preferences.
- Data complianceConduct AI compliance reviews during data collection, model training, and information provision to minimize legal risks.
- Search Enhancement Generation (RAG) technologyCombining retrieval and generation allows the model to handle longer contexts and be applied in complex scenarios.
EXAONE 3.5 project address
- Project official website:lgresearch.ai/blog/view
- GitHub repository:https://github.com/LG-AI-EXAONE/EXAONE-3.5
- HuggingFace model library:https://huggingface.co/collections/LGAI-EXAONE/exaone-35
- arXiv technical paper:https://arxiv.org/pdf/2412.04862
Application scenarios of EXAONE 3.5
- Chatbots and Customer ServiceAs the core of the chatbot, it handles customer queries and requests, providing 24/7 instant service.
- Language translation and cross-language understandingBased on bilingual capabilities, it assists in translation work and helps users from different language backgrounds communicate effectively.
- Content creation and editingThe model can generate creative copy, helping editors and writers expand their ideas and improve the efficiency and quality of content creation.
- Education and ResearchIn the field of education, it serves as a supplementary tool to help students learn languages and answer academic questions.
- Information retrieval and knowledge managementIn enterprises, it helps employees quickly find the information they need, improving work efficiency and decision-making quality.