In traditional software development, the spec-to-code gap is a well-known problem. Requirements are written in one document, code is written in another, and over time the two diverge until the spec is fiction and the code is the only source of truth.

For AI applications, this problem is 10x worse.

The AI Specification Problem

AI agents are non-deterministic. The same input can produce different outputs depending on context, model temperature, retrieved documents, and a dozen other variables. This makes traditional specification approaches — "given X input, produce Y output" — fundamentally inadequate.

What does a specification for an AI agent even look like?

Most teams skip this question entirely. They go straight to building: wire up some API calls, add a system prompt, test a few examples manually, and call it done. This is the "vibe coding" approach, and it works right up until it doesn't.

What We Mean by Spec-Driven

At Kapi, spec-driven development means three things:

1. Specifications Are Living Documents

A Kapi spec isn't a static document that gets written once and forgotten. It's a structured configuration that stays in sync with the running system:

agent: contract-reviewer
version: 2.3.0

behavior:
  primary_task: "Review contracts and extract key terms"
  must_not:
    - "Provide legal advice"
    - "Make recommendations on contract acceptance"
    - "Access documents outside the assigned scope"

  escalation:
    conditions:
      - "Contract value exceeds $1M"
      - "Non-standard indemnification clauses detected"
      - "Confidence score below 0.7"
    target: "legal-team"

evaluation:
  faithfulness:
    threshold: 0.85
    source: "retrieved_documents"
  completeness:
    required_fields:
      - "parties"
      - "effective_date"
      - "termination_clauses"
      - "payment_terms"
      - "liability_caps"

hitl:
  send_summary_email:
    pattern: "review"
    timeout: "4h"
    fallback: "escalate"
  flag_risk_clause:
    pattern: "approval"
    timeout: "24h"
    fallback: "hold"

This spec is both human-readable documentation AND machine-executable configuration. When you change the spec, the agent's behavior changes. When the agent's behavior drifts from the spec, you get an alert.

2. Specs Define Success Criteria

Every Kapi spec includes evaluation criteria with measurable thresholds. This transforms "does it work?" from a subjective judgment into a quantifiable metric:

Faithfulness > 0.85: At least 85% of the agent's claims must be grounded in retrieved documents
Completeness check: Every contract review must extract all required fields
Safety guardrails: PII must be detected and redacted before output
Cost ceiling: No single interaction should exceed $0.50 in API costs

These criteria aren't aspirational — they're enforced. Every interaction is evaluated, and failures trigger alerts or HITL escalation.

3. Specs Enable Drift Detection

The most insidious problem with AI systems is silent drift. The system works fine for weeks, then a model update changes behavior slightly, or a new document in your knowledge base introduces conflicting information, and suddenly your agent is producing subtly wrong outputs.

Spec-driven development catches this because:

Evaluation criteria run on every interaction (not just during testing)
Statistical quality scores are tracked over time
Significant deviations from baseline trigger alerts
A/B comparisons are built into the deployment pipeline

Traditional vs. Spec-Driven: A Comparison

Aspect	Traditional AI Dev	Spec-Driven (Kapi)
Requirements	Word doc, forgotten after week 1	Living spec, synced with system
Testing	Manual spot-checks	Automated eval on every interaction
Success criteria	"Looks good to me"	Quantified thresholds (faithfulness > 0.85)
Drift detection	Customer complaints	Automated alerts + dashboards
HITL policy	Ad-hoc	Formally specified per action
Code ownership	Locked in vendor	Full export, your infrastructure
Audit trail	Non-existent	Complete, queryable, compliant

The PM-Developer Handoff

One of the biggest benefits of spec-driven development is that it creates a clean contract between product managers and developers.

PMs define:

What the agent should do (and must not do)
Which actions need human approval
Quality thresholds and cost budgets
Escalation rules and fallback behavior

Developers implement:

The technical architecture that satisfies the spec
Integration with enterprise systems
Performance optimization
Deployment and monitoring infrastructure

The spec is the shared language. When a PM says "this agent needs to be more accurate," they can point to the faithfulness threshold and say "we need this above 0.90 instead of 0.85." When a developer says "this HITL pattern is creating bottlenecks," they can show the timeout data and propose moving specific actions to autonomous mode.

Getting Started

If you're building AI agents today without specifications, here's a pragmatic path to adoption:

Start with behavior boundaries. Before your next sprint, write down what your agent must NOT do. This is the most valuable part of any spec.
Add evaluation criteria. Pick one metric — faithfulness is a good start — and measure it on every interaction. You'll be surprised by the variance.
Define your HITL policy. For each action your agent takes in the real world, decide: autonomous, approval, review, or escalation? Write it down.
Track drift over time. Plot your evaluation scores weekly. When they change, investigate why.
Export and own. Make sure you can take your agent's code, specs, and test suites to any infrastructure. If you can't, you don't really own your AI.

The Future

As AI agents become more capable and more autonomous, the need for rigorous specification will only grow. We're moving from a world where AI is a tool (a human decides, AI executes) to a world where AI is an agent (AI decides, human supervises). In that world, specifications aren't optional overhead — they're the governance framework that makes autonomous AI systems safe and trustworthy.

Spec-driven development isn't just a better way to build AI. It's the only way that scales.

Kapi blueprints ship with living specifications, built-in evaluations, and full code export. Try it today.

Spec-Driven Development: A New Paradigm for AI