dacospace

AI Systems Lab

Design, break and defend enterprise AI systems: agents, RAG, evals, security, operations. 23 lessons · 24 incident files · 18 design questions.

Foundations

  1. 01An LLM is an unreliable dependencynondeterminism, pass^k, structured output, timeouts
  2. 02Workflows before agentsworkflow vs agent, augmented LLM, when not to build an agent

Agents

  1. 03Tools are an API for a modeltool design, idempotent tools, errors as observations, schemas
  2. 04Context is a budgetcontext engineering, context rot, lost in the middle, KV cache, compaction
  3. 05Loops that stopstep and cost budgets, stop conditions, runaway loops, durable execution
  4. 06Many agents, many failuresmulti-agent trade-offs, orchestrator-worker, MAST failure taxonomy
  5. 07Blast radius and human approvalleast privilege, irreversible actions, approval gates, audit logs

Security

  1. 08Prompt injectiondirect vs indirect injection, the lethal trifecta, why filters fail
  2. 09Where data leaks outexfiltration channels, markdown images, egress control, dual LLM, CaMeL
  3. 10MCP and the agent supply chaintool poisoning, rug pulls, rule-file backdoors, confused deputy, OAuth

Evals

  1. 11Error analysis before evalstrace review, failure taxonomy, golden datasets, criteria drift
  2. 12Judging the judgeLLM-as-judge, position and verbosity bias, aligning a judge with humans
  3. 13Evaluating agentsoutcome vs trajectory, pass^k, benchmark contamination
  4. 14Seeing inside an AI systemtracing, OpenTelemetry GenAI conventions, silent regressions

Knowledge

  1. 15RAG and its failure pointsretrieval-augmented generation, the seven failure points, retrieval metrics
  2. 16Better retrievalhybrid search, reranking, chunking, contextual retrieval
  3. 17Permissions follow the documentpermission-aware retrieval, oversharing, tenant isolation
  4. 18Grounding, citations and liabilityfaithfulness, citation checking, who is liable for the answer

Production

  1. 19Cost and latencyprompt caching, model routing and cascades, batch, semantic cache risks
  2. 20When the provider goes downrate limits, provider outages, fallbacks, degradation modes
  3. 21Guardrails and their limitsinput and output filters, classifiers, policy enforcement outside the model
  4. 22Governance for engineersEU AI Act, NIST AI RMF, ISO/IEC 42001, PII and retention

Capstone

  1. 23Capstone: an enterprise agent platformend-to-end design, review and defense

Incident files

Real incidents from primary sources. You get the facts, form a hypothesis, then reveal what actually happened.

Agents

  1. Cursor's support bot invents a policy and cancels its own customers (2025)hallucinated company policy, caused real subscription cancellations
  2. Project Vend: an AI runs a real shop and loses money on tungsten cubes (2025)self-reported research, autonomous business-decision failures over long horizons
  3. Replit's agent deletes a production database during a code freeze (2025)irreversible data loss, agent misreported its own blast radius

Evals

  1. Anthropic's three infrastructure bugs (2025)weeks-long, intermittent quality degradation across multiple models, root-caused to three overlapping infra bugs
  2. GPT-4o sycophancy rollback (2025)model-wide behavioral regression, rolled back within ~24 hours
  3. SWE-bench Verified contamination (2026)benchmark retracted by its own publisher after contamination audit

Knowledge

  1. Air Canada bereavement fare chatbot (2024)negligent misrepresentation finding, company held liable for chatbot statements
  2. Mata v. Avianca (2023)court sanctions, $5,000 fine against attorneys and firm
  3. NYC MyCity chatbot (2024)systematically wrong legal advice, kept online after being shown the errors
  4. Stanford study of legal AI research tools (2024)17-34% hallucination rate in commercial tools marketed as "hallucination-free"

Production

  1. Klarna's AI customer-service claims and partial walk-back (2024-2025)cost-optimized rollout, CEO-admitted quality regression, partial reversal
  2. McDonald's rolls back drive-thru AI ordering (2024)enterprise-scale reliability failure, full rollback after ~3 years
  3. OpenAI global outage, December 11, 2024~4h22m fleet-wide outage, ChatGPT, API, and Sora all down

Security

  1. A support ticket reads your whole Supabase database (2025)stored prompt injection leaking PII via SQL, self-reported
  2. A wiper prompt shipped inside a coding agent's own update (2025)supply-chain compromise, destructive AWS CLI payload, CVE-2025-8217
  3. Chevrolet of Watsonville chatbot "$1 Tahoe" (2023)prompt-injection induced "binding offer" for $1, viral but unenforced
  4. DPD chatbot swearing and self-disparaging poem (2024)guardrail disabled by an unrelated system update, exploited within hours
  5. EchoLeak: zero-click data theft from Microsoft 365 Copilot (2025)zero-click data exfiltration, CVE-2025-32711, CVSS 9.3
  6. GitHub MCP server: a public issue reads your private repos (2025)private-data exfiltration via public pull request, lethal-trifecta pattern
  7. Microsoft 365 Copilot "oversharing" (2024, ongoing)latent permission debt made discoverable at scale, delayed 40% of surveyed rollouts 3+ months
  8. Slack AI leaks private data through a public channel (2024)cross-channel data exfiltration via indirect prompt injection
  9. The approval gate that only checked half the cases (2025)remote code execution, CVE-2025-54130, CVSS 7.5
  10. The config file that writes vulnerable code on purpose (2025)hidden instructions in configuration, no special access required
  11. The tool you approved isn't the tool you're running (2025)MCP tool-definition compromise, CVE-2025-54136 (CVSS 8.8)

Design questions

Staff-level AI system design interviews. They also rotate into your daily Interview, every other day.

  1. Build a shared eval platform for many product teamsevals, observability, platformsenior · 45 min
  2. Choose between a single agent and a multi-agent systemagents, multi-agent, architecturesenior · 45 min
  3. Design a legal RAG assistant with trustworthy citationsknowledge, rag, citations, regulatedsenior · 45 min
  4. Design a long-running research agentagents, context-management, multi-agentsenior · 45 min
  5. Design a multi-tenant enterprise knowledge assistantknowledge, rag, multi-tenancy, hybrid-searchsenior · 45 min
  6. Design a sandbox for an unattended coding agentsecurity, agents, sandboxing, blast-radiussenior · 45 min
  7. Design AI governance and audit for a regulated companyproduction, governance, compliance, auditsenior · 45 min
  8. Design an LLM gateway with routing, caching and budgetsproduction, gateway, cost, caching, routingsenior · 45 min
  9. Design an MCP gateway for a company's internal agentssecurity, mcp, agents, oauthsenior · 45 min
  10. Design durable execution for a crash-prone agentagents, durability, idempotencysenior · 45 min
  11. Design multi-provider failover for an LLM-dependent productproduction, reliability, failoversenior · 45 min
  12. Design permission-aware search over enterprise documentsknowledge, rag, permissions, securitysenior · 45 min
  13. Design the AI platform for a 5,000-employee bankplatform, security, evals, knowledge, production, governancesenior · 60 min
  14. Diagnose and calibrate a miscalibrated LLM judgeevals, llm-judge, observabilitysenior · 45 min
  15. Make a refund-issuing support agent reliableagents, reliability, idempotencysenior · 45 min
  16. Review a RAG copilot for data exfiltration risksecurity, rag, exfiltration, prompt-injectionsenior · 45 min
  17. Safely roll out a model swap for a production agentevals, rollout, reliabilitysenior · 45 min
  18. Secure an email-reading support agent with refund authoritysecurity, agents, prompt-injectionsenior · 45 min