A prompt is the full control surface you expose to a pretrained model at inference time: system instructions, few-shot exemplars, retrieved context, tool schemas, and the user turn. Interviewers expect you to treat prompting as engineering — versioned, evaluated, and constrained — not as one-off copywriting.

• Separate instructions (stable) from data (untrusted) — never concatenate raw docs into the system role without labeling
• Prefer explicit output contracts: field names, enums, refusal behavior, citation format
• Put the most critical constraints first and last (primacy/recency); middle of long prompts is a dead zone
• Version prompts like code: hash, owner, eval suite, rollback path

┌─────────────────────────────────────┐
│ System: role, safety, output schema │
├─────────────────────────────────────┤
│ Developer: product rules, tools     │
├─────────────────────────────────────┤
│ Retrieved context (ACL-filtered)    │
├─────────────────────────────────────┤
│ Few-shot exemplars (optional)       │
├─────────────────────────────────────┤
│ User message + conversation memory  │
└─────────────────────────────────────┘
              ↓
         Tokenizer → LLM → decode

Context window size is a hard budget of tokens across system prompt, history, retrieval, tools, and generation headroom. Interviews dig into how you allocate that budget under latency and cost constraints — not just quoting a model's advertised max tokens.

1. Count tokens with the model's tokenizer (or a close proxy) — never estimate from characters alone for code
2. Reserve max_output_tokens before packing inputs; leave a safety margin
3. Summarize or drop oldest turns when history balloons; keep user goal + last decisions
4. Rank-and-truncate retrieval: score chunks, pack highest until budget, never silent overflow

Window = W tokens
1. Lock system + tools + schema          (immutable)
2. Reserve G for generation
3. Pack ranked retrieval until budget
4. Pack recent history (newest first)
5. Drop / summarize remainder

If query critical: retrieval > history
If multi-turn task: keep last tool results

Embeddings map text (or multimodal inputs) to dense vectors so semantic similarity becomes a distance or score. In interviews, embeddings show up as the backbone of retrieval, clustering, dedup, and semantic caching — not as a magic black box.

• Never mix embedding model versions in one index without a re-embed migration plan
• Chunk text before embedding — one vector for a 50-page PDF collapses local meaning
• Normalize vectors if your store assumes cosine/IP scoring
• Watch for asymmetric retrieval: query and document may need different prefixes or models

Query ──► Embed(q) ──► ANN (HNSW/IVF) ──► top-k ids
Docs  ──► Embed(d) ──► Vector index ────────────┘
                              │
                         Optional:
                    Cross-encoder rerank → top-n

Tokens are the billing and latency unit. English prose is roughly 1.3 tokens per word; code, JSON, and non-English scripts are denser. Capacity planning in interviews starts from QPS × (input + output tokens) × price.

• Always leave headroom for completion — truncated JSON is a production incident
• Output tokens often dominate latency and cost more than input on generative tasks
• Prompt caching (when available) amortizes large stable prefixes across requests
• Estimate daily $ = QPS × 86400 × (in+out)/1e6 × $/1M tokens

Model choice is a multi-objective decision: quality on your evals, p99 latency, $/query, context size, tool-calling reliability, data residency, and rate limits. Strong candidates propose a routing strategy, not a single forever model.

1. Define the task class and success metric before picking a model
2. Build a golden eval set; compare models on that — not Twitter benchmarks
3. Prototype with a strong model; distill/route down once quality floor is clear
4. Plan fallbacks: primary → secondary provider → smaller model with degraded features

Request → Classifier / heuristics
              │
     ┌────────┼────────┐
     ▼        ▼        ▼
  Cache hit  Small     Large
  (return)   (FAQ,     (complex,
              extract)   agents)
                 │
                 └──► Escalation if low confidence

Prompting and RAG change behavior at inference without weight updates. Fine-tuning (full, LoRA, preference/RLHF-style) changes weights. Interviews expect a clear decision tree: start with prompt + retrieval; fine-tune when format, tone, or domain language is stable and high-volume.

• Fine-tuning does not replace retrieval for up-to-date factual knowledge
• Data quality > data quantity; noisy labels lock in bad behavior
• Budget for eval, safety regression, and model versioning ops
• LoRA adapters can be swapped per tenant/task with lower storage cost

Modern interviews expand beyond chat: vision inputs, audio, and structured outputs (JSON Schema, tool calls, grammars). Treat modalities as different latency and safety surfaces with the same need for contracts and evals.

• Never trust model JSON without schema validation and typed parsing
• For vision+RAG: OCR or caption into text, then retrieve — or use native multimodal carefully
• Prefer constrained decoding when available over "please return JSON" alone
• Log raw vs parsed outputs for debugging schema regressions after model upgrades

LLM → raw tokens
        │
   Schema / grammar decode
        │
   Validate (zod/jsonschema)
        │
   ┌────┴────┐
   OK        Fail → retry w/ error
   │              or fallback path
   ▼
Typed object → business logic