Chapter
AI & FDE Interview Skills
Frameworks, tradeoff narratives, and whiteboard patterns for AI systems and FDE case interviews.
AI Systems Interview Framework
A strong 45–60 minute AI systems interview follows a repeatable arc: clarify → estimate → architecture → deep dives → production wrap. Memorize the skeleton so stress does not erase structure.
- Clarify: users, modalities, latency SLO, accuracy bar, data sources, privacy, online vs batch (5–8 min)
- Estimate: QPS, tokens in/out, cost/day, cache hit, peak load (5 min)
- High-level architecture: client → API → retrieval/agent → model → tools/DB (10–12 min)
- Deep dives: retrieval quality, tool safety, evals, failure modes (15–20 min)
- Production: observability, guardrails, rollback, 10× scale (5 min)
| Pitfall | Fix |
|---|---|
| Jump to agents first | Baseline RAG or single-call design |
| No eval story | Propose golden set immediately |
| Ignore ACLs | Filter chunks by identity |
| Optimize model before retrieval | Measure retrieval metrics first |
| Silent assumptions | State and confirm them |
Requirements → Numbers → Boxes & arrows
↓
Pick 2 deep dives
↓
Prod / failure / scaleTimeboxing
If the interviewer pulls you into a deep dive early, park the rest of the outline verbally so they know you have a full design.
Drive the conversation
Ask which deep dive they care about most: retrieval, agents, cost, or safety — then go deep there.
FDE Interview Framework
FDE cases blend product sense, integration realism, and stakeholder craft. Lead with customer framing, then MVP, spike plan, and rollout — technology serves that narrative.
- Frame the customer problem and constraints
- Map stakeholders and success metrics
- Scope an MVP with a Won't list
- Spike the hardest integration risk first
- Plan pilot rollout, training, and handoff
| Pitfall | Better move |
|---|---|
| Build before data access | Spike connectors + ACL first |
| Demo theater | Measurable pilot criteria |
| Ignore change management | Training + feedback loop |
| Over-customize | Config + productize patterns |
| No support plan | Handoff runbooks + owners |
Interview signal
Talk like you have sat in a customer war room: security questionnaire delays, incomplete APIs, and politics are first-class constraints.
Metric example
"Pilot succeeds if 30 agents resolve tickets 20% faster with ≥ citation click rate and zero ACL incidents."
Problem → Stakeholders → MVP
↓
Integration spike
↓
Pilot → Feedback → HandoffTradeoff Narratives
Senior interviews grade tradeoff quality. State options, criteria, recommendation, and what would change your mind. Avoid false dichotomies and "it depends" without criteria.
| Tradeoff | Option A | Option B | Decide with |
|---|---|---|---|
| RAG vs fine-tune | Fresh facts | Stable style | Change rate of knowledge |
| API vs self-host | Speed to market | Control/cost at scale | QPS + compliance |
| Agent vs workflow | Flexibility | Reliability | Task openness |
| Dense vs hybrid | Semantics | Exact IDs | Query mix |
| Big model always | Quality | Cost/latency | Router + evals |
- Name 2–3 real options, not a strawman
- Pick explicitly: "I'd choose X because Y"
- List risks of your choice and mitigations
- Invite interviewer constraints that could flip the decision
Phrase template
"Given SLO S and constraint C, I recommend A over B because metric M. If M flips or compliance requires residency, I'd switch to B."
Weak answer
Listing technologies without criteria. Interviewers hear "I memorized a stack," not "I can decide."
Whiteboard Patterns: RAG Systems
Most AI lab/FAANG design questions collapse to a RAG-shaped system. Practice drawing a clean default diagram you can extend under pressure.
Client → API Gateway → Orchestrator
│
┌────────────┼────────────┐
▼ ▼ ▼
Query rewrite Retriever LLM
│ (hybrid) │
│ │ │
│ Rerank │
│ │ │
└────────────┴────► Prompt
│
Answer+cites
Ingest: Connectors → Parse → Chunk → Embed → Index
(ACL metadata on every chunk)- Always draw ingest separately from query
- Label ACL filter on retrieve
- Mark eval harness and offline index quality jobs
- Add cache sitting beside the orchestrator when cost comes up
| Extension ask | What to add |
|---|---|
| Multi-tenant SaaS | Tenant_id filters, per-tenant indexes |
| Agents | Tool layer with AuthZ beside LLM |
| Global scale | Regional indexes, embedding CDN/cache |
| Strict citations | Verifier / NLI step before return |
Interview signal
Narrate as you draw. Silent diagramming loses communication points even if the boxes are right.
Numbers on the board
Annotate rough QPS, chunk size, top-k, and p99 targets on the diagram — it shows systems maturity.
Whiteboard Patterns: Agents
When the problem is open-ended (ops automation, research assistant, coding agent), draw a bounded plan–act loop with tool policy — not an unbounded brain.
User → Gateway → Agent runtime
│
Policy engine
│
┌─────────┼─────────┐
▼ ▼ ▼
Planner Tools API Memory
│ │ │
└──── observe ──────┘
│
step budget / stop
│
Final answer
Tools: search | ticket | code_exec (sandboxed)
Writes: require confirmation- Write max steps and timeout on the board
- Separate read tools from write tools visually
- Show audit log as a first-class box
- Include human-in-the-loop for irreversible actions
| Follow-up | Ready answer |
|---|---|
| How stop loops? | Dedup tool calls + N-step cap + critic |
| How prevent exfil? | Allowlist + arg policy + output redaction |
| How eval? | Task success suite + trajectory rubrics |
| How cost-control? | Router to smaller model; cache; step budget |
Interview signal
Propose the simplest agent that works. Complexity without a failure mode it solves is a red flag.
Do not
Draw five agents debating by default. Start with one agent + tools; add multi-agent only if asked or clearly needed.
Estimation & Communication
Back-of-envelope estimates and crisp communication separate staff-level AI candidates. Practice token/cost math and saying uncertainty ranges out loud.
| Cheat sheet | Rule of thumb |
|---|---|
| Tokens ≈ words × 1.3 | English; code denser |
| RAG context | Top-k × ~500–1K tokens/chunk |
| Daily cost | QPS × 86400 × (in+out)/1e6 × $/1M |
| Embedding cost | Usually << generation at scale |
| Cache win | 30–70% LLM spend reduction possible |
- State assumptions: "Assume 5 QPS peak, 2k in, 400 out…"
- Round aggressively; precision theater wastes time
- Compare options numerically when debating model tiers
- Summarize every 10 minutes: "So far we've decided X; next I'll dig into Y"
Communication pattern
Lead with the headline ("I'd do hybrid RAG with citations"), then justify. Do not hide the recommendation at the end of a monologue.
Check-ins
"Does this match the depth you want, or should I go deeper on retrieval?" — collaborative candidates get steered to their strengths.
Staff depth layer
Interview Skills — Staff Depth
AI systems interviews reward structured communication under uncertainty. Staff candidates clarify constraints, estimate with units, draw ingest vs query, pick one deep dive, narrate tradeoffs aloud, and close with risks and metrics — pacing like a 45–60 minute design review, not a lecture on transformers.
Anti-patterns → say instead
✕ Diving into transformers math
Burns time; most loops score product/systems judgment unless the role is research.
Say instead: One sentence on model choice rationale, then architecture, evals, and failure modes.
✕ Silent whiteboard
Interviewer can't score invisible reasoning; you look stuck when you're thinking.
Say instead: Talk in headlines: 'I'll clarify, estimate, draw boxes, then deep-dive retrieval.'
✕ Boiling every subsystem
Shallow coverage of 12 topics scores worse than one excellent deep dive.
Say instead: Propose the deep-dive menu; ask which they care about; go deep for 10–15 minutes.
✕ Ignoring the FDE/people layer
For FDE loops, pure tech without stakeholders and pilot metrics fails the rubric.
Say instead: Frame users, buyers, constraints, MVP, and success metrics alongside the architecture.
Prompt: 'Design permission-aware Q&A over Confluence + SharePoint for 25k employees.'
- 0–7 min: clarify users, ACL, latency, modalities, out of scope (no multi-agent MVP).
- 7–12 min: estimate QPS, chunks, embed cost, daily LLM $ — write numbers on board.
- 12–24 min: HLD — ingest webhooks, chunk/embed, ACL sync, query path with filter + cite.
- 24–40 min: deep dive — pick ACL freshness or eval harness based on interviewer nudge.
- 40–45 min: risks (injection, stale ACL, cost), observability, rollback, 10× scale note.
- Keep estimates within ~5 minutes; wrong constants OK if units and method are sound
- Say assumptions aloud: 'Assuming 20× peak-to-average…'
- Reserve 1 minute for a crisp closer: baseline RAG + ACL + golden set before agents
Pacing and explicit assumptions are scoring signals equal to technical correctness.
Cheat sheet
- Clarify → estimate → HLD → deep dive → risks/rollout
- State assumptions; invite correction
- Draw ingest vs query
- Offer deep-dive menu; go deep once
- Tradeoffs: quality / latency / cost / privacy
- Close with metrics, failure modes, rollback
- FDE: stakeholders + pilot KPIs + handoff