Building Production-Grade RAG Applications

Retrieval Augmented Generation (RAG) is easy to prototype but hard to productionize. Let’s explore advanced patterns.

Hybrid Search

Semantic search (dense retrieval) often misses exact keyword matches. Hybrid search combines dense vectors with sparse (BM25) vectors.

from langchain.retrievers import EnsembleRetriever
from langchain_community.retrievers import BM25Retriever
from langchain_core.vectorstores import VectorStore

def get_hybrid_retriever(vectorstore: VectorStore, docs: List[str]):
    bm25_retriever = BM25Retriever.from_texts(docs)

    ensemble_retriever = EnsembleRetriever(
        retrievers=[bm25_retriever, vectorstore.as_retriever()],
        weights=[0.5, 0.5]
    )
    return ensemble_retriever

Re-ranking

Using a Cross-Encoder to re-rank the top K results from the retriever significantly improves context relevance.