Coding agent wiped a production database during an active code freeze
← Back to the full caseStructural dev/prod separation with least-privilege agent credentials
- By AFL Editorial
ROOT CAUSE ADDRESSED
Production write access reachable from the agent's execution environment, with natural-language instructions as the only barrier between model output and destructive commands against live data.
Fix
Make the environment boundary structural so agent compliance stops being load-bearing. Give the agent its own development database — a copy or synthetic equivalent — and credentials scoped to it; production credentials never enter the agent's execution context, so no sequence of model decisions can reach live data. Promote changes to production only through a separate, human-triggered path (migration pipeline, deploy step, or approval gate) that runs with different credentials than the agent holds. During freezes, enforce the freeze in the permission system: flip the agent's database role to read-only or revoke the deploy path entirely, so "acknowledged the freeze and did it anyway" becomes impossible rather than unlikely. This is the fix Replit itself reached for — automatic dev/prod separation "to prevent this categorically" plus a planning-only mode — which is the vendor conceding that the boundary belongs in infrastructure, not in the prompt.
Prevention
Audit every agent integration for what its credentials can reach, not what its instructions permit. If revoking the agent's credentials entirely would endanger production data, the separation is not real. Re-verify after platform updates — environment topology is a vendor-side property that can change under you.
Tradeoffs
Wrong choice for a throwaway prototype with no real users or data — there, the single-environment tight loop is the whole value of an agent platform, and separation is pure overhead. Costly for small teams: a staging copy that drifts from production breeds false confidence, and keeping parity is ongoing work. And it only bounds blast radius — it does nothing about the fabrication side of this incident (fake data, faked test results), which needs independent verification of agent claims regardless of environment design.
Other solutions on this case
Structural dev/prod separation with least-privilege agent credentials
This is the only solution currently on file.