You handle AI pipeline failures gracefully by treating each failure mode as a distinct problem with its own fix — malformed output gets schema validation and a retry, slow tool calls get timeouts and circuit breakers, context overflow gets truncation or summarization before the call, and hallucinated fields get a confidence check with a path to a human. There's no single "error handling" layer that covers all of this. Teams that ship reliable AI features build four or five small, boring systems instead of one clever one.
That distinction matters because most teams don't build any of them until something breaks in production.
The common wrong approach
Most teams wrap the model call in a try/catch, log the exception, and call it done. It works in the demo because the demo uses clean inputs and a happy-path prompt. Then real traffic arrives: a customer email with three nested forwards, a PDF that OCR'd badly, a user request that doesn't map cleanly to any of your tool definitions. The model returns JSON with a trailing comma. It calls a tool with a hallucinated parameter that looks plausible but doesn't exist in your schema. It times out because the context window ballooned past what you tested with.
The try/catch catches the exception, sure. But "catch and log" isn't a strategy — it's a shrug. The pipeline stalls, the user sees a spinner or a generic error, and someone finds out three days later when a customer complains. We saw this exact pattern on an early version of Email Triage: the classifier worked fine until a vendor started sending auto-generated receipts with malformed HTML, and the whole triage queue backed up silently overnight.
The better approach
Design for each failure mode separately, at the point where it actually occurs.
Malformed output. Validate every model response against a schema before it touches downstream logic — Zod on the Node side, Pydantic on the Python side. Don't just check "is this valid JSON." Check field types, required fields, and value ranges. On failure, retry once with the validation error fed back into the prompt ("your last response was missing the `priority` field — return it again"). This single-retry-with-feedback pattern fixes the large majority of malformed outputs without a human ever seeing it.
Tool call timeouts. Set an explicit timeout on every tool call — shorter than you think you need, because a hung tool call blocks the whole turn. Wrap it in a circuit breaker so a failing downstream service (a flaky CRM API, a slow vector DB) doesn't take out every request in flight. On timeout, don't retry blindly — retry with backoff, cap it at two attempts, then fail the tool call explicitly and let the model decide what to do next, or fall back to a cached or degraded response.
Context window exceeded. This one you can catch before it happens. Track token count as you assemble the prompt, not after the API rejects it. When you're close to the limit, don't just truncate from the top — summarize older context, drop low-relevance retrieved chunks first, and keep the most recent turns and the system instructions intact. On Ghost Writer, we run a pre-flight token count on every draft-plus-research bundle and trigger a summarization pass at 80% of the limit, before the model ever sees a truncated mess.
Hallucinated required fields. This is the hardest one because the output looks structurally valid — right types, right shape, just factually wrong. The fix is a verification step: cross-check extracted fields against a source of truth where one exists (an order ID against your database, a date against the original document), and flag anything that can't be verified rather than passing it downstream as fact. On CV Matcher, we don't let an extracted "years of experience" figure feed into ranking until it's cross-referenced against the dates actually stated in the document.
Every one of these needs a ceiling. After N retries, stop retrying and escalate — to a fallback response, a degraded mode, or a human queue. Escalation isn't a failure of the pipeline; it's the pipeline working correctly.
Where this breaks
All of this is real engineering effort, and for a two-person team validating a rough idea, it's often not worth building yet. If you're at ten users and manually checking outputs anyway, spend your time on the product question, not a circuit breaker library. The risk is knowing when to stop deferring it — usually the moment you stop personally reading every output, or the moment a failure would reach a paying customer instead of you. Building this too early is wasted effort; building it too late is an incident.
The other honest limit: verification steps add latency and cost. Cross-checking every hallucination-prone field against a source of truth means an extra lookup, sometimes an extra model call. That's a real tradeoff against speed, and it's worth measuring rather than assuming.
Practical next step
This week, pick the one failure mode in your pipeline you've actually seen happen — check your logs, it's probably malformed output or a timeout — and add explicit handling for just that one: a schema validator with a single feedback-retry, or a timeout with a two-attempt backoff. Don't try to build all four systems at once. If you want a second pair of eyes on where your pipeline is most exposed, start a project with us and we'll walk through it together.