Skip to main content
Case study · AI implementation

The AI That Held

An AI report-review engine that holds up on a business owner's real reports where his own DIY chatbot attempts drifted.

Running on his real reports2026

AI implementation · Standards grounding · Structured output

The AI reviewer running on a real inspection, client name redacted: 136 photos auto-sorted into likely issues, needs-a-look, and cleared, scored against the owner's own rubric.
The AI reviewer on a real inspection, client name redacted. It sorted 136 photos into defects, needs-eyes, and cleared against the owner's own rubric, and surfaces every photo for the inspector's final say. No accuracy number is claimed yet.
Context

A home inspection company owner had already tried building this himself. Hundreds of hours into ChatGPT versions, all of them drifting. The rules got lost somewhere in one long chat and the model started improvising. That is how bolt-on AI usually ends, and it is why he came to me instead of trying another chatbot.

The build

I built the v1 engine. It reads his real inspection reports end to end and returns the review in his own scorecard format, the layout he used to fill in by hand at night. His standards are the grounding, so it flags things the way he does, not the way a generic chatbot would.

Why it holds

The model was never his problem. In my build the rules live outside it. Each check hands the AI only the slice of his standards that one question needs, so it cannot lose the plot the way a chat window does. A prompt is not a system. The machinery around the model is what makes AI dependable inside a real business.

The number I will not publish yet

I could quote an accuracy number today and it would look great. It would also be close to meaningless. The reports I have are the ones he already corrected, and testing on those flatters the engine. I won't publish an accuracy number until it's measured against reports the way his inspectors first submitted them. When a vendor hands you a suspiciously clean number, ask what they tested it on.

Status
Status registry
Running
The v1 engine, end to end on his real reports
Being proven
Catch rate, measured the honest way
Results

No accuracy number gets published until it is measured against reports the way inspectors first submitted them. Not before.

Stack

AI implementation · the client's own standards wired in as grounding · per-check retrieval · structured scorecard output