AB
AB
AiBoss

All Categories

project

RAG-FiT - Intel Labs launches open-source RAG framework for developing and enhancing large models.

RAG-FiT (formerly known as RAG Foundry) is an open-source framework developed by Intel Labs for fine-tuning large language models (LLMs) in retrieval augmentation generation (RAG) tasks. RAG-FiT...

What is RAG-FiT?

RAG-FiT (formerly known as RAG Foundry) is an open-source framework developed by Intel Labs for fine-tuning large language models (LLMs) to enhance their performance in retrieval augmentation generation (RAG) tasks. RAG-FiT is based on a modular design, supporting four main functional modules: data creation, training, inference, and evaluation. RAG-FiT helps users quickly create datasets suitable for RAG tasks, optimize models using Parametric Efficient Fine-Tuning (PEFT) techniques, and measure model performance using various RAG-specific evaluation metrics. The RAG-FiT framework is highly flexible and extensible, supporting a full range of RAG use cases from data selection and filtering to retrieval and suggestion generation, and is suitable for various application scenarios such as question answering systems and text generation.

Main functions of RAG-FiT

  • Data creation and processing:
    • Data loadingSupports loading datasets from Hugging Face Hub or local sources.
    • Data preprocessingThis includes data filtering, normalization, aggregation, information retrieval, and templated prompt generation.
    • Data storageThe processed data is saved in a consistent format to facilitate subsequent training and inference.
    • Flexible processing flowIt supports global operations (such as data aggregation and filtering) and local operations (such as retrieval of individual samples and text processing).
  • train:
    • Parameter Efficient Fine-Tuning (PEFT): Use techniques such as LoRA to efficiently fine-tune the model.
    • Training configurationIt supports custom training parameters such as learning rate, optimizer, and batch size.
    • Model pushThe trained model can be pushed to Hugging Face Hub.
  • reasoning:
    • Generate predictions: Generate prediction results on the processed dataset.
    • Batch inferenceIt supports batch processing of multiple input data, improving efficiency.
  • Evaluate:
    • Multi-dimensional assessmentIt supports multiple evaluation metrics, such as EM, F1, ROUGE, BERTScore, etc.
    • Custom assessmentUsers can easily implement custom evaluation metrics.
    • Global and Local AssessmentIt supports local evaluation of each sample and global evaluation of the entire dataset.

RAG-FiT Technical Principles

  • Search Enhancement:
    • Search mechanismThis involves retrieving contextual information related to the input question from external knowledge bases using retrieval tools. These tools include vector-based retrieval systems (such as Haystack and Qdrant) and other retrieval frameworks.
    • Context InjectionInjecting retrieved contextual information into the input of LLMs helps the model better understand the problem context, thereby generating more accurate and evidence-based answers.
  • Modular design:
    • Data processing moduleResponsible for loading, preprocessing, and saving data. Supports multiple data sources and flexible processing workflows.
    • Training moduleThe model is fine-tuned using PEFT technology and supports efficient training methods such as LoRA. The trained model can be saved or pushed to Hugging Face Hub.
    • Inference moduleGenerate prediction results on the processed dataset, supporting batch inference.
    • Evaluation moduleIt provides a variety of evaluation metrics to support multi-dimensional evaluation of the generated results.
  • Configuration driver workflow:
    • Hydra configuration toolThe Hydra configuration tool enables hierarchical configuration, supports overriding configuration values via command line, and facilitates remote job execution.
    • configuration fileEach module has a default configuration file, and users can customize workflows based on configuration files or command-line arguments.
  • Experiments and Evaluation:
    • Experimental environmentIt provides an end-to-end experimental environment that supports rapid prototyping and experimentation with various RAG technologies.
    • Multi-dimensional assessmentThe evaluation module assesses the accuracy of the generated results, as well as the relevance of the search results and the faithfulness and relevance of the generated content.

RAG-FiT project address

Application scenarios of RAG-FiT

  • Question and Answer SystemBased on the retrieval of external knowledge bases to enhance the language model, it provides users with more accurate and relevant answers, applicable to professional fields such as medicine and law.
  • Text generationIt combines the latest background information to generate high-quality text, such as news reports and creative writing, thereby improving the timeliness and accuracy of the content.
  • Knowledge Graph AugmentationIt retrieves entities and relationships from a knowledge graph, generates text consistent with the graph, and improves the accuracy and interpretability of knowledge representation.
  • Multilingual generationCross-language knowledge base retrieval, generating multilingual text to meet content generation needs in multilingual environments.
  • Document SummaryIt retrieves key information from documents to generate summaries, improving the accuracy and information coverage of summaries. It is suitable for scientific research, business and other fields.
Back