The Model Returned Invalid JSON: Three Causes and the Real Fix

Figures on this page were verified 31 August 2026 against the providers' own documentation. Pricing, context windows and rate limits change without notice, so confirm any number against the provider before you rely on it. Tell us if something here is out of date.

Before you write another parsing workaround, check the three causes in order: the output was truncated, the model wrapped the JSON in prose or a code fence, or you are asking it to produce JSON freehand when the provider offers a guaranteed mode. The third is the real fix. Regex repair of model output is a symptom of not using the feature that makes the problem impossible.

1. It is not invalid, it is incomplete

The most common cause of a JSON decode error is truncation. The model hit your output ceiling mid-object, so the braces never closed. Your parser reports a syntax error and points you at the wrong problem entirely.

Check the stop reason before you parse. stop_reason == "max_tokens" on Anthropic, finish_reason == "length" on OpenAI. If either is set, the output was cut off and no amount of parser tolerance will recover it. See why responses get cut off for the fix.

2. Valid JSON, wrapped in something else

Asked for JSON in a plain prompt, models frequently return this:

Sure, here is the JSON you asked for:

```json
{"name": "example", "value": 42}
```

Let me know if you need anything adjusted.

The JSON is perfectly valid. The prose and the code fence around it are not. Stripping fences with a regex works until the model returns two fenced blocks, or explains itself inside the fence, or uses a different language tag. Every patch holds until it does not.

3. The actual fix: stop asking nicely

Both providers offer ways to constrain output so that unparseable responses become structurally impossible rather than merely unlikely. Prompt instructions are a request; these are a guarantee.

• Structured outputs. Supply a schema and have the response validated against it. Nothing outside the schema comes back, so there is no prose to strip and no fence to remove.
• Tool use as a schema. Define a tool whose input schema is the shape you want, and read the arguments the model passes. This is often the cleanest route when you also want the model to decide whether to answer at all.

Where a strict mode is available, it is worth enabling. It changes the guarantee from “usually matches your schema” to “validates exactly”, which is the difference between an error you handle and an error that cannot occur.

Parse tool arguments, never string-match them

When you do read tool-call inputs, decode them properly:

# right
args = json.loads(tool_use.input) if isinstance(tool_use.input, str) else tool_use.input

# wrong: escaping varies between models and generations
if '"city": "London"' in raw_input:
    ...

Different models escape Unicode and forward slashes differently in serialised tool inputs. Code that string-matches the serialised form works on one model and breaks silently on the next, which makes it a genuinely nasty class of bug to track down after a migration.

Keep one defensive layer anyway

Even with a guaranteed mode, wrap the parse. Not because the schema will fail, but because truncation, a network cut, or an empty response will still reach your decoder.

if resp.stop_reason == "max_tokens":
    raise TruncatedOutput("output was cut off before the JSON closed")

try:
    data = json.loads(text)
except json.JSONDecodeError as e:
    log.error("unparseable model output", extra={"raw": text[:500], "err": str(e)})
    raise

Log the raw output, truncated to a sane length, whenever parsing fails. Without it you are debugging a JSON error with no idea what the JSON was, and the failure is often not reproducible on the next run.

Related

The truncation cause is covered in full in why the response got cut off. This page is part of our LLM API error reference.

Frequently asked questions

Why does the model wrap my JSON in prose or a code fence?

Because asking for JSON in the prompt is a request, not a constraint. Without a schema enforced at decode time the model is free to add a preamble, a markdown fence, or a closing explanation, and it sometimes will.

What is the reliable fix?

Structured outputs, or a tool and function schema, which constrain decoding so that invalid JSON cannot be emitted at all. This turns a probabilistic failure into an impossible one. Prompt wording alone never reaches that guarantee, however firmly it is phrased.

My JSON parses but data is missing. What happened?

You almost certainly hit max_tokens partway through the object. Check the stop reason before parsing, because a truncated array can still be valid JSON while silently dropping entries.