RAG Agent with Grounded Citations

Retrieval-augmented answering that cites source chunks and refuses when the context does not support an answer.

Pipeline

1. Ingest: split documents into ~300-800 token chunks with overlap; keep doc_id, title, url, and position metadata.
2. Embed and index chunks in a vector store (pgvector, SQLite-vec, Qdrant, etc.). Optionally add a keyword (BM25) index for hybrid search.
3. Retrieve top-k (e.g. 20) candidates, then rerank to the best 5.
4. Generate with the prompt below; chunks are labelled [1]..[n].
5. Validate: check every citation number exists; if the answer has no citations, treat it as unsupported.

Prompt

Answer the question using ONLY the sources below.
Cite sources inline like [2]. Every factual sentence needs a citation.
If the sources do not contain the answer, say "I don't know based on the provided sources."

Sources:
{{#each chunks}}[{{@index1}}] {{title}}: {{text}}
{{/each}}

Question: {{question}}

Tips

• Hybrid (vector + keyword) retrieval handles names, codes, and IDs far better than vectors alone.
• Evaluate with a small fixed question set and track answer correctness and citation accuracy over time.

Failure modes

• Citations that point to chunks that don't support the claim; validate cited IDs.
• Poor chunking splits answers across chunks.
• Stale index returns outdated facts; schedule re-indexing.

Inspiration

Real-world reports that inspired or relate to this blueprint, credited to Agent Use Cases:

• Make auditable rule-based decisions with RAG-written explanations (agentusecases.ai)

Markdown version: /blueprints/rag-with-citations.md

Related blueprints