Cross-Provider JSON Schema Compatibility: Anthropic vs OpenAI vs Gemini (2026)

Grammar-constrained decoding went mainstream in 2026. Every major LLM provider now guarantees that your model will emit structurally valid JSON — OpenAI with response_format, Anthropic via tool use, and Gemini through responseSchema. The days of wrapping your LLM call in a try-catch hoping the JSON parses are over.

Except they aren't. Not fully.

The problem has shifted: the question is no longer "will the model emit valid JSON?" It is "which slice of JSON Schema does each vendor actually honor?" Anthropic, OpenAI, and Gemini each enforce a different subset of the JSON Schema specification. The same Zod schema — perfectly valid spec — can be accepted on OpenAI, rejected on Gemini, and produce a runtime error on Anthropic. Production pipelines built for one provider quietly break on another.

This guide documents the actual compatibility matrix as of August 2026, with code examples for each provider.

---

Why the Drift Happens

Each provider built their structured output engine independently, targeting a different version of JSON Schema and making different tradeoffs for speed and correctness.

• OpenAI built against JSON Schema draft-07, then added pragmatic restrictions (no anyOf with null, no $ref loops, integer ceilings) to keep their FSM tractable at scale.
• Anthropic routes structured output through their tool use API, which uses a JSON Schema input_schema. The tool engine has its own parser — missing features like minimum/maximum bounds and some $ref patterns.
• Gemini uses responseSchema backed by a Google-internal proto-based schema compiler. It supports enum, object, and array reliably but has undisclosed limits on deeply nested schemas and additionalProperties.

The result: three enforcement models, three failure surfaces. And because each provider rejects invalid schemas at runtime — not at build time — the failures often surface in production, not during development.

---

The Compatibility Matrix

Key takeaway: The safe common subset is: object, array, string, number, boolean, enum, required, and nested properties. Anything beyond this needs provider-specific testing.

---

Provider-by-Provider Breakdown

OpenAI Structured Outputs (GPT-5 Series)

OpenAI's structured output API is the most widely documented. You pass your schema in response_format:

from openai import OpenAI
import json

client = OpenAI()

schema = {
    "type": "object",
    "properties": {
        "name": {"type": "string"},
        "score": {"type": "number"},
        "tags": {
            "type": "array",
            "items": {"type": "string"}
        }
    },
    "required": ["name", "score", "tags"],
    "additionalProperties": False
}

response = client.chat.completions.create(
    model="gpt-5",
    messages=[{"role": "user", "content": "Extract: Alice scored 92 with tags: senior, backend"}],
    response_format={
        "type": "json_schema",
        "json_schema": {
            "name": "extraction",
            "schema": schema,
            "strict": True
        }
    }
)

result = json.loads(response.choices[0].message.content)

OpenAI restrictions to know:

• anyOf with non-null types is banned under strict: true. To express optional fields, use "type": ["string", "null"] syntax, not anyOf.
• All object schemas must set "additionalProperties": false.
• All required arrays must list every property in the object.
• $ref cycles cause a schema validation error at request time.
• Integer values are capped at 2^53 — values beyond this silently truncate.

Anthropic (Claude Sonnet/Opus via Tool Use)

Anthropic does not have a json_schema response format. Structured output happens through tool use: you define a tool with an input_schema, then force the model to call it with tool_choice.

import anthropic
import json

client = anthropic.Anthropic()

tools = [{
    "name": "extract_result",
    "description": "Extract structured data from the text",
    "input_schema": {
        "type": "object",
        "properties": {
            "name": {"type": "string"},
            "score": {"type": "number"},
            "tags": {
                "type": "array",
                "items": {"type": "string"}
            }
        },
        "required": ["name", "score", "tags"]
    }
}]

response = client.messages.create(
    model="claude-sonnet-5",
    max_tokens=1024,
    tools=tools,
    tool_choice={"type": "tool", "name": "extract_result"},
    messages=[{"role": "user", "content": "Extract: Alice scored 92 with tags: senior, backend"}]
)

result = response.content[0].input

Anthropic restrictions to know:

• minimum, maximum, minLength, maxLength are accepted in the schema but not enforced — the model ignores them. Validate these yourself after the call.
• anyOf is supported in the schema but not grammar-constrained — the FSM cannot enforce union types at token level. Use it for documentation only.
• $ref with $defs works for shallow one-level references. Deeply nested $defs may cause schema parse errors.
• additionalProperties is not required to be false — Claude won't add extra keys by default.
• The schema lives in tool_input — parse response.content[0].input, not response.content[0].text.

Gemini (2.5 Flash / Pro via responseSchema)

Google Gemini's responseSchema is backed by a proto-based compiler. It's strict about type declarations and has the widest enum support but the most limited $ref support.

import google.generativeai as genai
import json

genai.configure(api_key="YOUR_API_KEY")

schema = {
    "type": "object",
    "properties": {
        "name": {"type": "string"},
        "score": {"type": "number"},
        "tags": {
            "type": "array",
            "items": {"type": "string"}
        }
    },
    "required": ["name", "score", "tags"]
}

model = genai.GenerativeModel("gemini-2.5-flash")
response = model.generate_content(
    "Extract: Alice scored 92 with tags: senior, backend",
    generation_config=genai.GenerationConfig(
        response_mime_type="application/json",
        response_schema=schema
    )
)

result = json.loads(response.text)

Gemini restrictions to know:

• $ref and $defs are unsupported — inline all schema definitions. Using $ref raises a schema validation error.
• Optional fields (properties not in required) should use the nullable: true extension rather than omitting from required, otherwise Gemini may still require them.
• additionalProperties is partially enforced — it accepts the key but doesn't always block extra properties.
• anyOf / oneOf support is limited — use explicit type arrays (["string", "null"]) for optional primitives.
• Deep nesting (5+ levels) occasionally fails with an undocumented schema compiler error.

---

What Actually Breaks: Real-World Failure Patterns

Pattern 1: The Optional Field (anyOf null)

You have an optional field that can be a string or null. The natural JSON Schema approach:

{
  "middle_name": {
    "anyOf": [{"type": "string"}, {"type": "null"}]
  }
}

OpenAI strict: true: Fails. Use "type": ["string", "null"] instead. Anthropic: Accepted but not constrained — model may omit the field or return a string. Gemini: Use "type": "string", "nullable": true — the Google extension.

Portable solution:

def optional_string():
    return {"type": "string", "nullable": True}  # Gemini
    # OR for OpenAI strict: use "type": ["string", "null"]
    # OR for Anthropic: accept either form, validate after

Pattern 2: Shared Zod Schema Across Providers

A common pattern in TypeScript pipelines is to define a single Zod schema and derive both validation and the JSON Schema from it:

import { z } from "zod";
import { zodToJsonSchema } from "zod-to-json-schema";

const schema = z.object({
  name: z.string(),
  score: z.number().min(0).max(100),
  tags: z.array(z.string()),
  category: z.enum(["A", "B", "C"]).optional(),
});

const jsonSchema = zodToJsonSchema(schema);

The output schema will include minimum: 0, maximum: 100 from .min(0).max(100). Anthropic ignores these. You must validate them yourself:

const result = await callAnthropic(jsonSchema);
if (result.score < 0 || result.score > 100) {
  throw new Error(`Score ${result.score} out of bounds`);
}

The .optional() on category generates anyOf: [{type: "string", enum: [...]}, {type: "null"}]. This fails on OpenAI strict mode. For OpenAI, strip optionality from the schema and include all properties in required.

Pattern 3: $defs Reference Reuse

To avoid repeating a schema, you might use $defs:

{
  "$defs": {
    "Address": {
      "type": "object",
      "properties": {
        "street": {"type": "string"},
        "city": {"type": "string"}
      },
      "required": ["street", "city"]
    }
  },
  "type": "object",
  "properties": {
    "billing": {"$ref": "#/$defs/Address"},
    "shipping": {"$ref": "#/$defs/Address"}
  },
  "required": ["billing", "shipping"]
}

OpenAI: Works — no cycles, so $ref resolves correctly. Anthropic: Works at one level of nesting. If Address itself contains $ref, it breaks. Gemini: Fails entirely. Inline both address objects manually.

---

Writing Portable Schemas: The Safe Subset

If you need the same schema to work across all three providers:

{
  "type": "object",
  "properties": {
    "name": {"type": "string"},
    "score": {"type": "number"},
    "active": {"type": "boolean"},
    "status": {"type": "string", "enum": ["pending", "active", "done"]},
    "tags": {
      "type": "array",
      "items": {"type": "string"}
    },
    "nested": {
      "type": "object",
      "properties": {
        "key": {"type": "string"}
      },
      "required": ["key"],
      "additionalProperties": false
    }
  },
  "required": ["name", "score", "active", "status", "tags", "nested"],
  "additionalProperties": false
}

Rules for portable schemas:

1. No anyOf, oneOf, allOf
2. No $ref or $defs
3. Every object includes "additionalProperties": false
4. Every property is listed in required (make nothing truly optional in the schema)
5. No minimum, maximum, minLength, maxLength — validate these in application code
6. No pattern (regex) constraints
7. No nesting deeper than 4 levels

---

Post-Call Validation Layer

Because each provider enforces a different subset, you need a validation layer regardless of which provider you use. Grammar-constrained decoding guarantees the JSON parses — it does not guarantee the values make sense.

from pydantic import BaseModel, Field, field_validator

class ExtractionResult(BaseModel):
    name: str = Field(min_length=1)
    score: float = Field(ge=0, le=100)
    tags: list[str]

    @field_validator("name")
    @classmethod
    def name_not_empty(cls, v):
        if not v.strip():
            raise ValueError("name cannot be blank")
        return v

# After any provider call:
import json
raw = call_any_provider(schema)
validated = ExtractionResult.model_validate(raw)

If validation fails — which happens at a rate of 7–15% for complex schemas on Anthropic when minimum/maximum constraints exist — repair the output and retry. Most bounds violations are slight (score: 101 instead of 100) and a single retry with the error message usually resolves them.

---

Provider Choice Guide

---

Fixing Schema Validation Errors

When your structured output call fails at the schema level (not the JSON parsing level), the error usually looks like:

• OpenAI: 400 Invalid schema: "anyOf" is not supported with strict mode
• Anthropic: invalid_request_error: tool input_schema is invalid
• Gemini: 400 Schema validation failed: Unsupported $ref

These are rejected before the model runs — your schema itself is malformed for that provider. Run your schema against aijsonmedic.com/json-schema-generator to validate it and identify provider-specific issues before deployment.

For errors that occur after the model runs — where the JSON structure is correct but values violate constraints — use aijsonmedic.com to repair and normalize the output, then re-validate.

---

FAQ

Q: Does OpenAI's strict mode guarantee the output is fully valid against my schema?

Yes — structural compliance is guaranteed. Value semantics (is the name actually a real name, is the score contextually appropriate) are not guaranteed. Schema shape is enforced; business logic is not.

Q: Why does Anthropic not support json_schema response_format?

Anthropic routes all structured output through tool use as an architectural choice. The benefit is that tools can be composed and re-used across turns. The tradeoff is that the API surface is more verbose.

Q: Can I use the same Pydantic model for Anthropic and OpenAI?

Yes, with caveats. Use model.model_json_schema() to generate the schema, then strip $defs references (inline them) and remove anyOf for optional fields before sending to Anthropic. A utility like instructor handles this automatically.

Q: What happens when Gemini hits an undisclosed schema limit?

You receive a 400 error at request time with a generic "schema validation failed" message. The fix is almost always to flatten nested objects or remove $ref references.

Q: Is grammar-constrained decoding the same as JSON mode?

No. "JSON mode" (e.g., response_format: {type: "json_object"}) only guarantees valid JSON — any valid JSON, including {}. Grammar-constrained decoding with a schema guarantees your schema shape. Always use a schema, not just JSON mode.

Q: Where should I validate bounds and value constraints that my LLM provider ignores?

In your application layer, immediately after the API call. Use Pydantic (Python) or Zod (TypeScript) to validate the parsed output before passing it downstream. Treat the provider's output as untrusted input to your validation layer.