DeepSeek R1 Distilled Models: Fix JSON Output Failures (All 6 Variants)

DeepSeek R1's distilled models are the most-run open-source reasoning models in 2026 — but their JSON output behavior differs sharply from the full 671B model, and from each other. If you're running Qwen-14B or Llama-70B distills via Ollama, vLLM, or llama.cpp and getting broken JSON, this guide covers every failure mode with exact fixes.

> Broken JSON from a distilled model? Paste it into the AI JSONMedic repair tool — it handles all DeepSeek-specific patterns including <think> bleed, truncation, and Python boolean leakage.

---

The Distillation Context: Why It Matters for JSON

DeepSeek released six distilled variants of R1 in January 2025: 1.5B, 7B, 8B, 14B, 32B, and 70B. These are dense LLMs (Qwen2.5 and Llama3 backbones) trained via supervised fine-tuning on reasoning traces from the full R1 model.

The critical gap: the May 2026 R1-0528 update added proper JSON output enforcement and function calling to the full 671B model — but the distilled checkpoints were not updated. The 1.5B through 70B distills remain at their original January 2025 weights.

What this means in practice:

---

Failure Mode 1: <think> Block Bleed Into JSON

This is the most common distilled-model JSON failure. The reasoning block (<think>...</think>) bleeds into the output before or around the JSON.

What it looks like:

<think>
The user wants a product object with name, price, and category fields. I should return...
</think>
{"name": "Widget Pro", "price": 29.99, "category": "hardware"}

Or worse — the JSON ends up inside the think block:

<think>
Here is the JSON output:
{"name": "Widget Pro", "price": 29.99}
</think>

The fix:

import re, json

def extract_json_from_deepseek(raw: str) -> dict:
    # Strip <think>...</think> block (greedy match handles multiline)
    cleaned = re.sub(r'<think>.*?</think>', '', raw, flags=re.DOTALL).strip()

    # Extract JSON object or array from the remainder
    json_match = re.search(r'(\{.*\}|\[.*\])', cleaned, re.DOTALL)
    if not json_match:
        raise ValueError(f"No JSON found after think-strip: {cleaned[:200]}")

    return json.loads(json_match.group(1))

For TypeScript:

function extractJsonFromDeepSeek(raw: string): unknown {
  // Remove think block
  const cleaned = raw.replace(/<think>[\s\S]*?<\/think>/g, '').trim();

  // Extract JSON
  const match = cleaned.match(/(\{[\s\S]*\}|\[[\s\S]*\])/);
  if (!match) throw new Error(`No JSON in output: ${cleaned.slice(0, 200)}`);

  return JSON.parse(match[1]);
}

---

Failure Mode 2: Truncated JSON (Token Budget Exhaustion)

Distilled models burn tokens on reasoning. On a 14B model with a 4096-token context, a complex reasoning chain can leave only 200–400 tokens for the actual output. The result: truncated JSON.

{
  "analysis": {
    "summary": "The query involves three main components...",
    "components": [
      {"name": "auth", "priority": "high", "dependencies": ["db",

The model hit its token limit mid-array.

The fix:

1. Increase max_tokens — distilled models need a separate budget for the reasoning phase. A 14B with complex tasks needs 6000–8000 total tokens minimum.

1. Use a repair library as fallback:

from jsonrepair import repair_json

def safe_parse(raw: str) -> dict:
    # Strip think block first
    cleaned = re.sub(r'<think>.*?</think>', '', raw, flags=re.DOTALL).strip()

    try:
        return json.loads(cleaned)
    except json.JSONDecodeError:
        repaired = repair_json(cleaned)
        return json.loads(repaired)

1. Separate reasoning from output — use a two-turn pattern: first turn for reasoning, second turn with "Based on your analysis, output ONLY the JSON object with no explanation" and a lower temperature.

---

Failure Mode 3: JSON Inside Markdown Fences

Distilled models frequently wrap JSON in code fences even when you explicitly ask for raw JSON:

Here's the structured output you requested:

json

{"result": "success", "data": {"count": 42}}

Let me know if you need anything else!

The fix:

def strip_markdown_json(raw: str) -> str:
    # Remove think block
    text = re.sub(r'<think>.*?</think>', '', raw, flags=re.DOTALL).strip()

    # Extract from code fence
    fence_match = re.search(r'

(?:json)?\s([\s\S]?)\s*```', text)

if fence_match:

return fence_match.group(1).strip()

# Fall back to raw JSON extraction

json_match = re.search(r'(\{[\s\S]\}|\[[\s\S]\])', text)

return json_match.group(1) if json_match else text

---

## Failure Mode 4: Schema-Valid but Pydantic/Zod Rejection

Distilled models produce JSON that parses correctly but fails schema validation. Common patterns:

- **Missing required fields** — model omits fields it's uncertain about
- **Wrong types** — numeric strings instead of numbers (`"42"` instead of `42`)
- **Null-filled required fields** — model outputs `null` rather than omitting
- **Extra reasoning fields** — model adds `"reasoning": "..."` fields not in your schema

**The fix — correction prompt loop:**

python

from pydantic import BaseModel, ValidationError

import anthropic # or your local inference client

def parse_with_correction(prompt: str, schema: type[BaseModel], client, max_retries: int = 2):

messages = [{"role": "user", "content": prompt}]

for attempt in range(max_retries + 1):

response = client.chat(messages=messages)

raw = response.content

# Extract JSON from DeepSeek output

cleaned = strip_markdown_json(extract_think_block(raw))

try:

data = json.loads(cleaned)

return schema.model_validate(data)

except (json.JSONDecodeError, ValidationError) as e:

if attempt == max_retries:

raise

# Add correction context

messages += [

{"role": "assistant", "content": raw},

{"role": "user", "content": f"That output failed validation: {e}\n\nReturn ONLY the corrected JSON object, no explanation."}

]

---

## Failure Mode 5: Python Boolean / None Leakage

When distilled models are fine-tuned on Python code-heavy datasets, they can output Python literals instead of JSON:

json

{"active": True, "deleted": False, "user": None, "tags": ['admin', 'user']}

This is valid Python but invalid JSON (`True` → `true`, `None` → `null`, single-quoted strings).

**The fix:**

python

from jsonrepair import repair_json

def normalize_python_literals(text: str) -> str:

"""repair_json handles True/False/None and single-quoted strings."""

return repair_json(text)

Or manually for performance-sensitive paths:

python

import re

def fix_python_literals(text: str) -> str:

text = re.sub(r'\bTrue\b', 'true', text)

text = re.sub(r'\bFalse\b', 'false', text)

text = re.sub(r'\bNone\b', 'null', text)

# Single-quoted strings are harder — use repair_json for those

return text

---

## Running DeepSeek Distills with JSON Mode via Ollama

Ollama (0.7+) supports grammar-constrained decoding for JSON via the `format` parameter:

python

import ollama

import json

response = ollama.chat(

model='deepseek-r1:14b',

messages=[{'role': 'user', 'content': 'Extract the product details as JSON: ...'}],

format='json', # Enables grammar-constrained JSON output

options={

'num_predict': 2048, # Extra budget for reasoning phase

'temperature': 0.1,

}

)

Still strip think block even with format='json' — Ollama enforces

JSON structure but doesn't strip the reasoning prefix on all versions

raw = response['message']['content']

result = extract_json_from_deepseek(raw)

**vLLM JSON mode:**

python

from openai import OpenAI

client = OpenAI(base_url="http://localhost:8000/v1", api_key="none")

response = client.chat.completions.create(

model="deepseek-ai/DeepSeek-R1-Distill-Qwen-14B",

messages=[{"role": "user", "content": prompt}],

response_format={"type": "json_object"},

max_tokens=4096, # Account for reasoning tokens

)

vLLM strips think blocks when response_format=json_object is set

result = json.loads(response.choices[0].message.content)

---

## The Full Production Pattern

Combine all fixes into a single robust handler:

python

import re, json

from jsonrepair import repair_json

from pydantic import BaseModel, ValidationError

def deepseek_json_parse(raw: str, schema: type[BaseModel] | None = None) -> dict:

"""

Robust JSON extraction from DeepSeek R1 distilled model output.

Handles: think blocks, markdown fences, truncation, Python literals.

"""

# Step 1: Strip think block

text = re.sub(r'[\s\S]*?', '', raw, flags=re.DOTALL).strip()

# Step 2: Extract from markdown fence if present

fence = re.search(r'``(?:json)?\s*([\s\S]*?)\s*``', text)

if fence:

text = fence.group(1).strip()

else:

# Extract raw JSON object/array

match = re.search(r'(\{[\s\S]\}|\[[\s\S]\])', text)

if match:

text = match.group(1)

# Step 3: Parse with repair fallback

try:

data = json.loads(text)

except json.JSONDecodeError:

try:

data = json.loads(repair_json(text))

except Exception as e:

raise ValueError(f"Could not parse DeepSeek output: {e}\nRaw (first 500): {raw[:500]}")

# Step 4: Schema validation (optional)

if schema:

return schema.model_validate(data).model_dump()

return data

```

For cases where the repair library can't fix the output automatically, paste it into the AI JSONMedic JSON repair tool — it handles all five failure modes above with a visual diff showing exactly what changed.

---

Which Distill for Which Use Case?

---

FAQ

Q: Should I use format="json" in Ollama or add JSON instructions to the prompt?

Both — format parameter enables grammar-constrained decoding (structural guarantee), but the prompt instructions tell the model what schema to produce. Use both: format='json' in the API call + "Return a JSON object with keys: X, Y, Z" in the prompt.

Q: Why does Qwen-14B produce valid JSON in testing but truncate in production?

Production requests often have longer system prompts and conversation history. The 14B's 4096 default context fills up faster. Set num_ctx: 8192 in your Ollama Modelfile or max_model_len=8192 in vLLM, and increase max_tokens to give the reasoning phase room.

Q: My distilled model outputs JSON correctly 90% of the time. Is a repair layer worth it?

Yes. At 10% failure rate with 1000 daily requests, that's 100 hard failures per day. A try/repair_json/retry wrapper adds ~2ms overhead per call and reduces failures to near-zero.

Q: Does the local LLM JSON debugging guide cover DeepSeek-specific patterns?

It covers Ollama/vLLM/llama.cpp debugging in general. This guide adds DeepSeek-specific patterns: think block extraction, the distilled-vs-full model gap, and the correction prompt loop for schema validation failures.

---

Related

• Local LLM JSON Output Debugging — Ollama, vLLM, llama.cpp
• Fix Invalid JSON
• LLM Structured Output Repair Guide
• JSONPath Tester — test JSONPath queries against your model output
• AI JSONMedic Repair Tool — instant repair for any DeepSeek JSON failure