One Bad Tuesday
How would you even know this system had stopped working?
- Reliability
- Observability
- Model drift
- Evaluations
- Ownership
Interactive prototype
Open in a new tab →If the prototype doesn't load here, open it in a new tab.
The situation
An AI diligence assistant has run cleanly for three months. On a Tuesday, a permissions change, a dead source URL, a model update, an edited prompt, and a stalled CRM sync all land within a few hours.
Nothing errors. The system keeps producing confident, well-formatted, increasingly wrong answers.
The obvious answer
Add monitoring and alerts, so someone gets paged when the system breaks.
The question underneath it
This class of failure never throws. The design problem is defining what 'correct' means well enough that silent degradation becomes a visible signal.
What I'm paying attention to
Silent failure modes
The dangerous outputs are the fluent ones.
Input health
Monitor the sources, not just the endpoint.
Evaluations
A small golden set, run continuously, catches drift a dashboard won't.
Change control
Prompts are production code and deserve versioning.
Ownership
Every AI system needs a name attached to it after launch.
Recovery
Rolling back should take minutes, not an investigation.
My take
A prototype tells you what happens when everything works. Engineering tells you what happens when the API changes, the data goes stale, someone edits the prompt, or Tuesday happens.
The goal isn't to eliminate failure. It's to make failure visible, bounded, and recoverable.
Related writing
The Demo Is Not the System
Substack essay — Coming Soon
Explore another experiment
The AI Draft
You have a limited AI budget. What actually deserves to get funded?
Enter The AI Draft →ArchitectureBuild / Buy / Bend
You need the capability. You don't necessarily need to build the software.
Make the Call →AutonomyBlast Radius
How much power should you actually give your AI agent?
Configure the Agent →Office Hours with Chanel
Working through a version of this in your own company?
Private AI office hours for leaders who need to turn messy AI ideas into clear strategy, useful workflows, or safe internal agents — with senior-engineer judgment in the room.