# Structured Outputs and the Validation Trap > How to get a model to return clean JSON without paying the retry tax. The minimum schema design, the validation pattern, and the three schema failures that quietly leak budget. **Published:** 2026-04-07 **Reading time:** 7 minutes **Author:** Bernardo Campos (Founder, 21xVentures) **Canonical:** https://21xventures.com/blog/structured-outputs-validation-trap/ --- "Just have the model return JSON" sounds like it should be easy. In production, it's where a lot of features go to bleed budget. The model returns invalid JSON, you retry. The retry returns valid JSON with a wrong field type, you retry. The third retry returns the right type with a hallucinated enum value, you retry once more. Now this one user action has cost you four model calls, your P95 latency is shot, and the support ticket about it lands the next morning. Clean structured outputs are not a prompting problem. They are a *schema design + validation + retry-discipline* problem. Get those three right and you can stop fighting your model. ## Don't ask for "JSON" — define the schema Almost every modern provider has native structured-output support: OpenAI's `response_format` with a JSON Schema, Anthropic's tool-use as JSON enforcement, Gemini's controlled generation, and equivalents for open-source serving stacks via JSON-mode and grammar-constrained decoding. If you are still prompting "respond in JSON, do not include backticks," you are leaving free quality on the table. Define the schema, pass it to the provider, and let the decoder enforce it. The model will deviate dramatically less. This is also where most teams discover their schema is poorly designed: the moment the constraint becomes load-bearing, the model starts failing in interesting and informative ways. ## The validation pattern Even with native structured outputs, you still need a parse-and-validate layer on your side. The model can satisfy the schema and still return nonsense. 1. **Parse.** Use Pydantic (Python), Zod (TypeScript), or your stack's equivalent. Don't roll your own schema validator — you will get it wrong. 2. **Validate against business rules.** "Status must be one of X, Y, Z" is schema. "If status is X, the refund amount must be > 0" is a business rule. Run both. 3. **Repair locally if you can.** Trailing commas, mismatched casing on enum values, an extra wrapper object: fix these in code, don't pay for another model call. 4. **Escalate exactly once.** If parse + repair fails, retry the model call *once*, with the validation error included in the prompt ("your previous response failed validation: X. Please fix and return only the corrected JSON"). After one retry, give up — return a graceful error to the user. No call should fire more than twice. 5. **Log every parse failure.** First-class metric. A spike in parse-failure rate is almost always a prompt regression or a model snapshot drift. ## The three schema failures ### 1. Over-constrained schema "Return a list of exactly 7 tags, each between 2 and 4 words." The model can satisfy each of those individually but the conjunction trips it. Worse: when the right answer is genuinely 4 tags, the model invents three more. Fix: relax the constraints to ranges (1-10 tags), or split into two steps (generate, then prune). Don't force false precision. ### 2. Free-text fields that should be enumerated `{ "status": "string" }` almost guarantees you'll get "approved", "Approved", "APPROVED", and "approve" — sometimes in the same week. The downstream code that switches on the status is broken in three subtle ways and you have no idea. Fix: use enums in the schema. `{ "status": "approved" | "rejected" | "pending" }`. Native structured-output decoders will enforce this; even if yours doesn't, the post-parse validation will fail fast in code you control. ### 3. Required fields the model rarely has A required `confidence_score` field that the model is supposed to fill in by introspection. Models are *bad* at calibrating confidence. The values you get back are essentially noise dressed up as data. Fix: don't require what the model can't reliably produce. If you need a confidence signal, derive it from log-probs or from a separate classifier call. If a field is genuinely optional, mark it optional and accept null. ## The retry trap (cross-reference) This is the single most common way structured outputs leak cost. - **Cap retries at one.** Two model calls maximum per user action, ever. - **Track parse-failure rate as a percentage of total calls.** Alert when it crosses 5%. - **When a deploy spikes the failure rate, roll back immediately.** The eval set may have missed it; the failure-rate metric won't. - **Make the failure mode graceful.** "We had trouble with that — try a smaller request" is a better user experience than a hung request, an empty response, or a 500. ## Tools, briefly None of these are silver bullets. All of them are better than DIY string parsing. - **Pydantic v2** (Python): schema + validation + JSON Schema export in one. Pair with provider-native structured outputs. - **Zod** (TypeScript): same idea, idiomatic for JS/TS. Works with the OpenAI/Anthropic/Gemini SDKs. - **Outlines / Instructor / BAML** (cross-language): higher-level wrappers around the same pattern. Useful if you're switching providers often. - **Grammar-constrained decoding** (open-source models via vLLM, llama.cpp, etc.): the strongest form of enforcement, but only available on stacks you control. ## Logging structured-output health Three numbers belong on a dashboard from day one: - **Parse-failure rate** — % of calls where the first response failed validation. - **Repair-success rate** — of those, how many were fixed in code without a second model call. - **Retry-success rate** — of the rest, how many succeeded on the one retry. Below 60% means your retry prompt isn't carrying the validation error properly. These are cheap to compute, expensive to learn from incidents instead. ## Close Structured outputs are not a place to be clever. The pattern is boring and the wins are large: define the schema, let the decoder enforce it, validate against business rules, repair what you can in code, retry once, log everything, fail gracefully. Teams that follow this pattern stop fighting their model. Teams that don't end up paying for that fight in dollars, latency, and weekend debugging. --- **Related** - [Cost Ceilings for AI Features](https://21xventures.com/blog/cost-ceilings/) — structured-output retries are the biggest budget leak - [Evals Before Features](https://21xventures.com/blog/evals-before-features/) — parse-failure rate is a first-class rubric check - [Logging for LLM Systems](https://21xventures.com/blog/logging-for-llm-systems/) — the three dashboard metrics - [Prompt Versioning Without the Hairball](https://21xventures.com/blog/prompt-versioning/) — a parse-failure spike is almost always a prompt regression **Contact:** hello@21xventures.com