Retrieval-Augmented Generation grounds model answers in external evidence: ingest documents → index → retrieve relevant snippets → generate with citations. Interviews reward separating the ingest path from the query path and naming what quality means at each stage.

INGEST                         QUERY
Docs → Parse → Chunk           User query
         ↓                          ↓
    Embed + metadata           Embed / expand query
         ↓                          ↓
    Vector + keyword index  →  Retrieve → Rerank
                                   ↓
                              Prompt + generate
                                   ↓
                              Answer + citations

• Baseline: BM25-only or dense-only before adding agents
• Measure retrieval separately from generation (you cannot fix bad retrieval with a smarter prompt forever)
• Propagate document ACLs into every chunk metadata field
• Design for corpus refresh: deletes, updates, version pins

Chunking decides the atomic unit of retrieval. Too large → noisy context and diluted embeddings. Too small → lost coherence and broken tables. Interview answers should name size, overlap, and structure awareness.

1. Choose chunk size relative to embedding model and generation budget (often 200–800 tokens)
2. Add 10–20% overlap for narrative docs; less for highly structured specs
3. Keep metadata: source id, title, section path, page, ACL, updated_at
4. Special-case tables, code, and slide decks — generic splitters destroy them

Document
  └─ Parent section (large)
       ├─ Child chunk A  ← embed & retrieve
       ├─ Child chunk B
       └─ Child chunk C
Hit on B → expand to parent → send parent to LLM

Ingest quality caps RAG quality. Parsing PDFs, HTML, Confluence, tickets, and code into clean text with structure is an engineering problem: layout, OCR, boilerplate removal, and incremental updates.

• Idempotent ingest: content hash → skip unchanged; upsert/delete by stable doc id
• Preserve provenance: URI, revision, checksum for every chunk
• Strip boilerplate (headers/footers) or embeddings cluster on noise
• Queue-based pipeline with dead-letter for poison documents

Connector → Fetch → Parse → Normalize
                              ↓
                     Chunk → Embed → Upsert index
                              ↓
                     Update ACL / tombstone deletes
                              ↓
                     Emit metrics (lag, fail rate)

Vector indexes approximate nearest-neighbor search over dense embeddings. Choices (pgvector, Pinecone, Weaviate, Milvus, OpenSearch) trade recall, latency, filtering, ops burden, and cost. Interviews care about ANN algorithms and metadata filters more than brand names.

• Pre-filter vs post-filter ACLs: pre-filter can hurt recall; post-filter wastes candidates
• Plan re-index when embedding model changes — treat it like a schema migration
• Monitor recall@k offline when you change efSearch / nprobe for latency
• Multi-tenant: separate indexes or mandatory tenant_id filters on every query

Embed(query)
     ↓
ANN candidates (k*overfetch)
     ↓
Metadata filters (ACL, type, time)
     ↓
Optional sparse merge (BM25)
     ↓
Rerank → top-n chunks

Hybrid search combines lexical (BM25/sparse) and dense retrieval. Lexical wins on exact IDs, error codes, and rare tokens; dense wins on paraphrases. Most production RAG systems need both.

1. Retrieve top-k from sparse and dense independently
2. Fuse with RRF (reciprocal rank fusion) or weighted score normalization
3. Overfetch before fusion if filters will cull results
4. A/B fuse weights on nDCG — do not hardcode 0.5/0.5 forever

Dense ranks:  d1,d2,d3,...
Sparse ranks: s1,s2,s3,...

score(doc) = Σ 1 / (k + rank_i(doc))

Sort by fused score → candidates for rerank

Rerankers (cross-encoders or LLM-as-judge) reorder a short candidate list with higher precision. Context selection then packs the best evidence into the prompt under token limits — quality, not quantity.

• Rerank only overfetched candidates — never the whole corpus
• Deduplicate near-identical chunks before packing to save tokens
• Diversity: MMR or coverage heuristics so all facets of a multi-part question appear
• Drop chunks below a score threshold; empty context should trigger abstention

Corpus (millions)
    → Hybrid retrieve (~100)
    → Rerank (~10)
    → Pack / compress (~3–5)
    → LLM answer

Grounding means claims in the answer are supported by retrieved evidence. Citations make that checkable. Faithfulness evals catch when the model invents details not in context — a top FAANG RAG deep-dive topic.

1. Prompt: answer only from provided context; say "I don't know" otherwise
2. Require inline citation markers mapped to chunk ids
3. Post-validate: each cited id exists in retrieved set; optional NLI entailment check
4. UI: show quote highlights so users can audit without trusting the model