Writing
9 published pieces across 4 outlets, grouped by the failure they are about rather than by date. Each one opens with an incident that actually happened.
A model sounds right and is wrong
Nothing about a fluent answer indicates whether it is grounded. The system has no way to say it does not know, so it invents, and the invention arrives in the same tone as the truth.
The Unanswerable Test Suite: How to Test AI Systems That Sound Right When They Are Wrong
Argues that testing an AI system needs a suite built from questions it should refuse, not just ones it should answer, opening with a consulting firm's AI-generated report caught citing sources that never existed. Proposes tracking refusal rates directly and anchoring factual claims to source documents with deterministic checks, not another model's judgement. (opens in a new tab)Dataconomy / 1 Jun 2026Design your AI agents around how they fail, not what they can do
Argues agentic systems should be designed from their failure modes, opening with the Moffatt v. Air Canada chatbot case. Introduces FRAME: failure classification, recovery logic, awareness boundaries, monitoring hooks, and an escalation protocol.Framework: FRAME (opens in a new tab)Rows disappear and nothing errors
A row cap, a join that multiplies, a key that does not match. The pipeline succeeds, the dashboard renders, and the number is quietly wrong for weeks.
The Quiet Failure Mode of AI Features: When Your Cached Outputs Go Stale
Traces a semantic-search bug to a bulk update that bypassed the save hook, leaving cached embeddings stale with no visible error. Lays out a three-layer defence: mark on change, recompute in the background, and run a reconciliation sweep that catches whatever the first two miss. (opens in a new tab)AITechTrend / 4 Jun 2026The JVS Audit: Why Your Cross-Platform Dashboards Are Quietly Lying to You
A tool-agnostic audit for cross-platform dashboards, opening with the 2020 case where nearly 16,000 COVID results were lost to a spreadsheet row limit. JVS is Join, Validate, Standardise: catch silent row multiplication, measure key match rates, and hold one authoritative definition per metric.Framework: JVS Audit (opens in a new tab)A timeout tells you nothing, so the retry charges twice
The request may never have arrived, or it may have completed and the response been lost. Both look identical to the caller, and only one of them is safe to repeat.
The script runs against the wrong host
Data scripts get a fraction of the review that application code gets, and they are the only code that cannot be rolled back by deploying the previous version.
The test passes and proves nothing
A green run is evidence only if you know what it ran against. Coverage counts lines executed, not assertions made, and a suite nobody can distrust is a suite nobody checks.
The same instant gets two answers
Two clocks, two zones, a schedule declared against local time, a day boundary compared the wrong way. Every one of them is invisible until a date arrives that nobody tested.
The system you did not need
A cluster bought for traffic that never arrived is not neutral. It is a thing to operate, secure, upgrade and explain, paid for while the product is still unproven.