How to move an AI pilot to production.

Before any code, answer five questions in writing:

1. What's the one workflow we'd ship first if we had to pick one this week?
2. What does "ready" look like for that workflow? (Eval pass rate, cost ceiling, latency SLO, rollback path.)
3. What's the worst thing this system can do to a real user or to our data, and what stops it?
4. What gets cut from scope to hit the date?
5. Who owns the runbook on day 22?

If the team can't answer all five, the pilot isn't stuck on Claude. It's stuck on definition. Fix that before writing code.

Twenty to fifty regression cases representing the workflow's actual job. Inputs that look like real production traffic. Outputs scored against a deterministic rubric or a judge model with a held-out gold set. Eval suite goes into CI, runs weekly, blocks deploys on regression. If your team can't write the cases, the workflow isn't defined yet. This is the single highest-leverage step.

• Scoped MCP tools. Typed inputs, typed outputs, scoped OAuth, kill switch. Write tools route through a reviewer queue.
• Auth boundary in code. The model never holds raw credentials. Your server enforces row-level security.
• Output validation. Structured outputs validated against a schema. Failures retry with a corrective prompt, then fail closed.
• Token guardrails. Per-session and per-day caps, fallback to a cheaper model on context overflow.
• Observability. Every call logged with prompt, response, user, timestamp, latency, cost. Piped to your existing stack.

Ship the one workflow with the clearest ROI. Canary rollout: 5%, 25%, 100%, gated on eval pass rate and cost-per-call ceiling. Runbook checked into the repo. Dashboard live. Pair-programming handoff so your engineers own the build. Resist the urge to ship the second workflow in parallel: scope creep is the most common reason pilots stall.

Once one workflow is live and stable for two weeks, the second one slots into the same observability stack, the same auth boundary, the same eval CI. The marginal time to ship workflow N+1 is a fraction of the first. This is the leverage you wanted from the AI investment, but you only get it by sequencing.

When the team has missed the original launch date by 30+ days, when scope keeps growing instead of shrinking, when the engineers can't agree on "ready", that is the signal. An outside Systems Decision Audit replaces that stalemate with one documented ship, fix, redesign, or stop decision.