fastrag

---

Build and explore efficient retrieval-augmented generative models and applications

:round_pushpin: Installation • :rocket: Components • :books: Examples • :red_car: Getting Started • :pill: Demos • :pencil2: Scripts • :bar_chart: Benchmarks

fastRAG is a research framework for efficient and optimized retrieval augmented generative pipelines, incorporating state-of-the-art LLMs and Information Retrieval. fastRAG is designed to empower researchers and developers with a comprehensive tool-set for advancing retrieval augmented generation.

Comments, suggestions, issues and pull-requests are welcomed! :heart:

:mega: Updates

• 2023-12: Gaudi2, ONNX runtime and LlamaCPP support; Optimized Embedding models; Multi-modality and Chat demos; REPLUG text generation.
• 2023-06: ColBERT index modification: adding/removing documents; see IndexUpdater.
• 2023-05: RAG with LLM and dynamic prompt synthesis example.
• 2023-04: Qdrant DocumentStore support.

Key Features

• Optimized RAG: Build RAG pipelines with SOTA efficient components for greater compute efficiency.
• Optimized for Intel Hardware: Leverage Intel extensions for PyTorch (IPEX), 🤗 Optimum Intel and 🤗 Optimum-Habana for running as optimal as possible on Intel® Xeon® Processors and Intel® Gaudi® AI accelerators.
• Customizable: fastRAG is built using Haystack and HuggingFace. All of fastRAG's components are 100% Haystack compatible.

:rocket: Components

For a brief overview of the various unique components in fastRAG refer to the Components Overview page.

:round_pushpin: Installation

Preliminary requirements:

• Python 3.8 or higher.
• PyTorch 2.0 or higher.

To set up the software, clone the project and run the following, preferably in a newly created virtual environment:

pip install .

There are several dependencies to consider, depending on your specific usage:

# Additional engines/components
pip install .[intel]               # Intel optimized backend [Optimum-intel, IPEX]
pip install .[elastic]             # Support for ElasticSearch store
pip install .[qdrant]              # Support for Qdrant store
pip install .[colbert]             # Support for ColBERT+PLAID; requires FAISS
pip install .[faiss-cpu]           # CPU-based Faiss library
pip install .[faiss-gpu]           # GPU-based Faiss library
pip install .[knowledge_graph]     # Libraries for working with spacy and KG

# User interface (for demos)
pip install .[ui]

# Benchmarking
pip install .[benchmark]

# Development tools
pip install .[dev]

License

The code is licensed under the Apache 2.0 License.

Disclaimer

This is not an official Intel product.