OpenAI JSON Mode vs Structured Outputs: Handling JSON Parse Errors

Developers using the OpenAI API frequently encounter one of two scenarios: they enable JSON mode and still get a SyntaxError: Unexpected end of JSON input, or they implement structured outputs and discover edge cases where the response still needs manual handling. This guide explains both modes, their failure points, and how to build a reliable JSON parsing pipeline.

What is OpenAI JSON Mode?

JSON mode is enabled by setting response_format: { type: "json_object" } in your API call. When enabled, GPT-4 and GPT-4o are constrained to produce a JSON object as their response.

const response = await openai.chat.completions.create({
  model: "gpt-4o",
  response_format: { type: "json_object" },
  messages: [
    { role: "system", content: "Return user data as JSON." },
    { role: "user", content: "User: Alice, age 30" }
  ]
});

What JSON mode guarantees: The response will be a JSON object (not a JSON array, not a string, not markdown). What JSON mode does NOT guarantee: Schema correctness, completeness (truncation still happens), or that JSON.parse() will always succeed.

Most Common JSON Mode Failure: Truncation

The most frequent cause of JSON parse failures with JSON mode is token limit truncation. When your JSON response exceeds max_tokens, OpenAI cuts the response off mid-stream:

{"users": [{"id": 1, "name": "Alice"}, {"id": 2, "name": "Bo

This is syntactically invalid JSON — JSON.parse() throws immediately. The fix is twofold:

1. Set max_tokens high enough for your expected output size (at least 2-3x the average response length)
2. Use a repair fallback for edge cases that still slip through

// Defensive parsing with repair fallback
async function safeJsonParse(content) {
  try {
    return JSON.parse(content);
  } catch {
    const res = await fetch('https://aijsonmedic.com/api/repair', {
      method: 'POST',
      headers: { 'Content-Type': 'application/json' },
      body: JSON.stringify({ input: content })
    });
    const { output, valid } = await res.json();
    if (valid) return JSON.parse(output);
    throw new Error('JSON could not be repaired');
  }
}

What are OpenAI Structured Outputs?

Structured outputs, introduced in August 2024, go further than JSON mode. You provide a JSON Schema and OpenAI guarantees the response matches that schema exactly:

const response = await openai.chat.completions.create({
  model: "gpt-4o-2024-08-06",
  response_format: {
    type: "json_schema",
    json_schema: {
      name: "user_profile",
      strict: true,
      schema: {
        type: "object",
        properties: {
          name: { type: "string" },
          age: { type: "integer" },
          email: { type: "string" }
        },
        required: ["name", "age", "email"],
        additionalProperties: false
      }
    }
  },
  messages: [...]
});

What structured outputs guarantee: The response will match your schema exactly (when it completes successfully). What structured outputs do NOT guarantee: Completion before max_tokens — truncation is still possible.

JSON Mode vs Structured Outputs: Side-by-Side

Error Patterns to Handle in Both Modes

1. Truncated JSON

// Input truncated at max_tokens
{"data": [1, 2, 3, 4

This is the most common failure. The AI JSONMedic repair engine closes unclosed brackets, arrays, and strings to recover a valid (if incomplete) JSON value.

2. Markdown Code Fences

Even with JSON mode enabled, some models occasionally wrap responses in markdown:

json

{"result": "ok"}

AI JSONMedic strips these automatically.

3. Trailing Commas

{"name": "Alice", "active": true,}

Not valid JSON but common in AI output. Removed automatically.

4. NaN and Infinity

{"score": NaN, "max": Infinity}

Not valid JSON. Converted to null by the repair engine.

Building a Reliable JSON Pipeline

Here's the complete production pattern for both JSON mode and structured outputs:

import OpenAI from 'openai';

const openai = new OpenAI();
const REPAIR_API = 'https://aijsonmedic.com/api/repair';

async function repairJson(content: string): Promise<unknown> {
  const res = await fetch(REPAIR_API, {
    method: 'POST',
    headers: { 'Content-Type': 'application/json' },
    body: JSON.stringify({ input: content }),
  });
  const { output, valid, issues } = await res.json();
  
  if (!valid) {
    throw new Error(JSON repair failed. Issues: ${issues?.length ?? 0});
  }
  
  console.log(Repaired ${issues?.length ?? 0} JSON issues);
  return JSON.parse(output);
}

async function callOpenAIWithJsonMode(prompt: string) {
  const response = await openai.chat.completions.create({
    model: 'gpt-4o',
    max_tokens: 2000,  // set generously
    response_format: { type: 'json_object' },
    messages: [
      { role: 'system', content: 'Return only valid JSON. No markdown.' },
      { role: 'user', content: prompt }
    ],
  });

  const content = response.choices[0].message.content ?? '';
  
  // Primary: direct parse
  try {
    return JSON.parse(content);
  } catch {
    // Fallback: AI JSONMedic repair
    return repairJson(content);
  }
}

When to Use Which Mode

Use JSON mode when:

• You don't know the exact schema upfront
• You're doing exploratory extraction
• You want simple setup without schema definition

Use structured outputs when:

• You know exactly what fields you need
• You're integrating into typed systems (TypeScript, Pydantic)
• Schema violations would break downstream code
• You need the highest reliability for production

Always use a repair fallback when:

• Any AI-generated JSON goes directly to JSON.parse()
• You're handling long responses with high token counts
• Token budget is tight relative to expected output size

Python Implementation

import json
import requests
from openai import OpenAI

client = OpenAI()
REPAIR_API = "https://aijsonmedic.com/api/repair"

def safe_parse_openai_json(content: str) -> dict:
    """Parse OpenAI JSON mode output with repair fallback."""
    try:
        return json.loads(content)
    except json.JSONDecodeError:
        response = requests.post(
            REPAIR_API,
            json={"input": content},
            timeout=10
        )
        result = response.json()
        if result["valid"]:
            return json.loads(result["output"])
        raise ValueError(f"Could not repair JSON: {content[:100]}")

def get_structured_data(user_input: str) -> dict:
    response = client.chat.completions.create(
        model="gpt-4o",
        max_tokens=1000,
        response_format={"type": "json_object"},
        messages=[
            {"role": "system", "content": "Return JSON only. No markdown."},
            {"role": "user", "content": user_input}
        ]
    )
    return safe_parse_openai_json(response.choices[0].message.content)

Frequently Asked Questions

Q: Does JSON mode guarantee no parse errors?

A: No. Truncation at max_tokens is the most common remaining failure. Always wrap JSON.parse() in try/catch and use a repair fallback.

Q: Can I use structured outputs with streaming?

A: Not yet as of mid-2026. Structured outputs require non-streaming responses. Use JSON mode with streaming, then repair truncation on the final assembled string.

Q: My JSON repair returns a different structure than expected — why?

A: When JSON is truncated, the repair engine closes the structure at the truncation point. The data may be incomplete. Increase max_tokens to prevent truncation rather than relying on repair to fill in missing data.

Q: Is the AI JSONMedic repair API free?

A: Yes. The current API is free with a 2MB input limit and no authentication required. A Pro tier with higher limits is planned.

Summary

• JSON mode reduces JSON parse failures but does not eliminate them
• Structured outputs are more reliable but still subject to truncation
• Always use try/catch around JSON.parse() for AI-generated content
• Use the AI JSONMedic API as a repair fallback in production
• Set max_tokens generously to minimize truncation risk
• Use the AI JSONMedic online tool to manually inspect and repair problematic responses during development

For more context, see also: Fix ChatGPT JSON Errors, JSON Syntax Errors Explained, Validate JSON API Responses, and the OpenAI JSON Mode repair tool to fix malformed OpenAI responses interactively.