MiniRAG - HKU Launches New RAG System for Efficient Deployment of Small Language Models

What is MiniRAG?

MiniRAG, developed by the University of Hong Kong, is a novel Retrieval-Enhanced Generative (RAG) system designed for the efficient deployment of Small Language Models (SLMs) in resource-constrained environments. MiniRAG achieves this through two key technologies: a semantically aware heterogeneous graph indexing mechanism that combines text fragments with named entities, reducing reliance on complex semantic understanding; and a lightweight topology-enhanced retrieval method that uses graph structures for efficient knowledge discovery without requiring advanced language capabilities. In experiments, MiniRAG demonstrates performance comparable to Large Language Models (LLMs) while requiring only 25% of the storage space. MiniRAG provides comprehensive benchmark datasets to evaluate the performance of lightweight RAG systems in real-world device scenarios, promoting the development of efficient and private RAG systems in resource-constrained environments.

MiniRAG's main functions

• Efficient knowledge retrievalBased on a unique heterogeneous graph indexing mechanism, it can quickly and accurately retrieve the most relevant knowledge from a large amount of data, providing support for generating accurate answers.
• Lightweight model compatibilityDesigned specifically for small language models (SLMs), it significantly reduces the demand for computing resources and storage space without sacrificing too much performance, making it more suitable for running on resource-constrained devices.
• Strong reasoning abilityWith the help of a query-guided reasoning path discovery mechanism, MiniRAG can handle complex, multi-step reasoning tasks and effectively find the correct answer even when SLMs have limited semantic understanding capabilities.
• Highly adaptableMiniRAG can adapt to different application scenarios, including instant messaging, personal content management, local document retrieval, etc., providing users with personalized information retrieval and generation services.

MiniRAG's technical principles

• Semantic-aware heterogeneous graph indexing mechanism:
  • Text block node (Vc)Preserve the contextual integrity of the original text, directly participate in the retrieval stage, and ensure that the most relevant contextual content is identified.
  • Entity node (Ve)Key semantic elements extracted from text blocks, such as events, locations, time references, and domain-specific concepts, are used for anchoring semantic understanding.
  • Entity-Entity Join (Eα)Captures semantic relationships, hierarchical structures, and spatiotemporal dependencies between named entities.
  • Entity-Text Block Connectivity (Eβ): Connect named entities and their corresponding contexts, maintaining contextual relevance and semantic coherence.
  • Semantic description of edgesDescriptions generated based on language models provide explicit relational context for connected nodes, enhancing the semantic understanding capabilities of graphs.
• Lightweight topology-enhanced retrieval methods:
  • Query semantic mappingUsing entity extraction and lightweight sentence embedding models, user-input queries are aligned with graph index data to identify text blocks relevant to the query.
  • Query-driven inference path discoveryBased on the intelligent query guidance mechanism, a reasoning path is constructed in the heterogeneous graph, taking into account the semantic relevance between the query and entity nodes and the structural coherence between entities.
  • Topology-enhanced graph retrievalBy combining embedding-based similarity search and graph topology information, we identify high-quality entity-entity connections that are relevant to the query and extract logically related inference chains based on a path discovery mechanism.
  • Search related text blocksBased on entity-text block connections, we collect text blocks related to the reasoning path, calculate the semantic similarity between the input query and the text blocks and their edge descriptions, and finally select the most relevant text blocks for subsequent generation tasks.
  • Fusion Enhancement GenerationThe retrieved key relationships and optimal text blocks are integrated with the previously determined answer nodes based on the designed fusion strategy to construct a comprehensive and structured input representation, which is used in the final enhanced generation process.

MiniRAG's project address

• GitHub repository:https://github.com/HKUDS/MiniRAG/
• arXiv technical paper:https://arxiv.org/pdf/2501.06713

MiniRAG Application Scenarios

• Instant MessagingQuickly retrieve historical chat records, generate intelligent replies, and provide knowledge assistance.
• Personal content managementSearch personal notes, generate content summaries and reminders, and manage your schedule.
• Local document retrievalIt retrieves local document content, integrates information from multiple documents, and generates intelligent summaries.
• Privacy-sensitive applicationsA question-and-answer service that securely manages medical and financial information and protects privacy.
• Edge device applicationsProvides rapid response and intelligent services on devices such as smartwatches and smart speakers.