Case study · FDE · 35 min

Multi-Tenant AI Customization Without Forks

Support per-customer prompts, tools, and retrieval sources without maintaining one-off code forks for each deployment.

FDEmulti-tenantconfig

Interview opener

Every customer wants bespoke prompts and workflows.

1.When does customization become an unmaintainable fork?
2.How do you productize patterns across tenants?
3.Who pays for permanent snowflakes?

Strong signals

Config vs code boundary
Shared core + tenant overlays
Commercial conversation on true custom work

Hit 0/3

Weak signals (avoid)

Fork the repo per customer
Say yes to everything

Requirements

50 tenants; 10 want custom tone; 5 need custom tools; all share core product
Config-driven customization; no per-tenant git branches
Isolation: data, cache, secrets, rate limits per tenant
Upgrade: core improvements roll to all within 1 week

Back-of-envelope Estimation

Metric	Calculation	Result
Tenants	50	50
Config versions/tenant	avg 4	200 configs
Fork cost if branched	50 × 2 eng-days/upgrade	100 days/release
Config approach	1 core + overlays	~3 days/release
Savings	100 − 3	~97 eng-days/release

API Design

Endpoint	Method	Description
PUT /v1/tenants/{id}/config	PUT	Overlay: prompts, tools allow-list, branding
GET /v1/tenants/{id}/config/diff	GET	Show divergence from core defaults
POST /v1/tenants/{id}/validate	POST	Run tenant eval suite before activate

Data Model

Entity	Key fields	Notes
tenant_config	tenant_id, version, json_overlay	JSON Schema validated
tool_registry	tool_id, schema, risk_level	Global catalog
tenant_tools	tenant_id, tool_id, enabled	Allow-list join

High-level Design

Overlay composition

Request + tenant_id
   → load core defaults
   → merge tenant overlay (deep merge with deny-lists)
   → resolve tools allow-list
   → bind retrieval to tenant indexes
   → execute pipeline
Admin UX edits overlay → validate schema + eval → activate version

Deep Dive: What Belongs in Config

Safe: system prompt snippets, tone, retrieval filters, enabled tools
Unsafe as config: arbitrary code, unrestricted HTTP tools, shared cache keys
Productize repeated custom work after 3 similar tenants
Charge enterprise SKU for custom tools to fund maintenance

Bottlenecks at Scale

Config sprawl — lint unused overlays quarterly
One tenant's heavy tool breaks shared workers — isolate executors
Upgrade breaks overlay assumptions — contract tests per tenant suite

Failure Modes & Monitoring

SLO	Target	Alert
Config activate success	99%	Warn on validate fail spike
Cross-tenant data leak	0	Red-team continuous
Core upgrade adoption	< 7 days	Track lagging tenants
Custom tool errors	< 1%	Per-tenant dashboards

AI & FDE Interview Room