JSON Schema for AI Agents: Production Validation Guide (2026)

JSON Schema for AI Agents: Production Validation Guide

AI agent frameworks — LangGraph, CrewAI, PydanticAI, AutoGen — produce JSON at every step. Tool call arguments, observations, memory entries, chain-of-thought scratchpads, final outputs. All of it is JSON, and any of it can be malformed.

In production, unvalidated agent JSON is a liability. A tool call with a wrong argument type silently fails. An observation parsed incorrectly corrupts the agent's next decision. An output that doesn't match the expected schema breaks the downstream consumer.

This guide covers how to validate JSON at every point in an agent pipeline — with practical patterns for LangGraph, CrewAI, and PydanticAI.

Why Agent JSON Fails Differently

Agent JSON fails for different reasons than a simple one-shot LLM call:

Multi-hop accumulation errors: An error in step 3's JSON observation becomes the input to step 4. By step 7, you have compounded errors that are hard to trace back to their origin. Tool call argument drift: As conversation history grows, the model's tool call arguments can drift away from the original schema — especially for tools called repeatedly with varying inputs. Memory corruption: Agents that write to memory in JSON format can corrupt that memory with malformed entries, affecting all subsequent reads. Context length truncation: Long-running agents hit context limits. When the model is forced to truncate, JSON payloads are often the first casualty — resulting in incomplete objects. Model fallback behavior: When the primary model hits rate limits and falls back to a smaller model, JSON quality drops significantly. The fallback model may not have the same structured output compliance.

The Three Validation Points

Every agent pipeline has three points where JSON validation pays off:

1. Tool Call Input Validation

Before a tool executes, validate that its arguments match the expected schema. Catch errors before they propagate into tool side effects.

from pydantic import BaseModel, field_validator
from typing import Optional
import json

class WebSearchArgs(BaseModel):
    query: str
    max_results: int = 10
    filter_date: Optional[str] = None

    @field_validator("query")
    @classmethod
    def query_not_empty(cls, v: str) -> str:
        if not v.strip():
            raise ValueError("Search query cannot be empty")
        return v.strip()

    @field_validator("max_results")
    @classmethod
    def results_in_range(cls, v: int) -> int:
        if v < 1 or v > 50:
            raise ValueError("max_results must be 1-50")
        return v

def validate_tool_call(tool_name: str, raw_args: dict) -> dict:
    validators = {
        "web_search": WebSearchArgs,
        # add other tools...
    }
    if tool_name not in validators:
        return raw_args  # unknown tool — pass through

    model = validators[tool_name]
    validated = model.model_validate(raw_args)
    return validated.model_dump()

2. Tool Output Validation

Tool outputs are observations that the agent reasons over. If they're malformed, the agent reasons from incorrect data. Validate outputs before they enter the agent's context:

from pydantic import BaseModel
from typing import List, Optional

class SearchResult(BaseModel):
    url: str
    title: str
    snippet: str
    published_date: Optional[str] = None

class SearchResults(BaseModel):
    results: List[SearchResult]
    total: int
    query_used: str

def validate_search_output(raw_output: str | dict) -> SearchResults:
    if isinstance(raw_output, str):
        try:
            data = json.loads(raw_output)
        except json.JSONDecodeError:
            # Repair broken JSON before validation
            data = repair_json(raw_output)  # see repair section below

    return SearchResults.model_validate(data)

3. Final Output Validation

The agent's final output is the most critical validation point — it's what your downstream systems consume. Validate it strictly:

from pydantic import BaseModel, model_validator
from typing import List
from enum import Enum

class Confidence(str, Enum):
    HIGH = "high"
    MEDIUM = "medium"
    LOW = "low"

class AgentOutput(BaseModel):
    answer: str
    confidence: Confidence
    sources: List[str]
    reasoning_steps: int

    @model_validator(mode="after")
    def answer_not_empty(self) -> "AgentOutput":
        if not self.answer.strip():
            raise ValueError("Agent answer cannot be empty")
        return self

    @model_validator(mode="after")
    def sources_required_for_high_confidence(self) -> "AgentOutput":
        if self.confidence == Confidence.HIGH and len(self.sources) == 0:
            raise ValueError("High confidence claims require at least one source")
        return self

LangGraph Integration

In LangGraph, state is a typed dictionary that flows between nodes. Add validation at node boundaries:

from langgraph.graph import StateGraph, END
from typing import TypedDict, List
from pydantic import BaseModel
import json

class AgentState(TypedDict):
    messages: List[dict]
    tool_results: List[dict]
    final_answer: str | None

class ResearchOutput(BaseModel):
    key_findings: List[str]
    confidence: str
    sources: List[str]

def research_node(state: AgentState) -> AgentState:
    # Call LLM, get structured output
    response = call_llm(state["messages"])

    # Validate before storing in state
    try:
        validated = ResearchOutput.model_validate_json(response)
        state["tool_results"].append(validated.model_dump())
    except Exception as e:
        # Log validation failure, use fallback
        print(f"Research output validation failed: {e}")
        state["tool_results"].append({
            "key_findings": [],
            "confidence": "low",
            "sources": [],
            "validation_error": str(e)
        })

    return state

# Build graph with validation at each node
graph = StateGraph(AgentState)
graph.add_node("research", research_node)
graph.add_node("synthesize", synthesize_node)  # similar validation
graph.add_edge("research", "synthesize")
graph.add_edge("synthesize", END)

The key pattern: each node validates its output before writing to state. Downstream nodes never see unvalidated data.

CrewAI Integration

CrewAI agents produce task outputs that feed into the next task. Validate them with Pydantic:

from crewai import Agent, Task, Crew
from pydantic import BaseModel
from typing import List

class MarketAnalysis(BaseModel):
    market_size_usd_billion: float
    growth_rate_pct: float
    top_competitors: List[str]
    key_risks: List[str]

    @field_validator("market_size_usd_billion")
    @classmethod
    def size_must_be_positive(cls, v: float) -> float:
        if v <= 0:
            raise ValueError("Market size must be positive")
        return v

# Use output_pydantic for automatic Pydantic validation
analysis_task = Task(
    description="Analyze the JSON tools market for 2026",
    expected_output="Structured market analysis",
    output_pydantic=MarketAnalysis,  # CrewAI validates output automatically
    agent=analyst_agent,
)

crew = Crew(
    agents=[analyst_agent],
    tasks=[analysis_task],
)

result = crew.kickoff()
# result.pydantic contains the validated MarketAnalysis object
analysis: MarketAnalysis = result.pydantic

CrewAI's output_pydantic parameter does the validation for you when the task completes. If the agent's output fails validation, CrewAI retries automatically (up to the configured max retries).

PydanticAI Integration

PydanticAI is built around Pydantic validation natively — it's the tightest integration:

from pydantic_ai import Agent
from pydantic import BaseModel
from typing import List

class ExtractedEntities(BaseModel):
    people: List[str]
    organizations: List[str]
    locations: List[str]
    dates: List[str]

agent = Agent(
    "anthropic:claude-sonnet-4-6",
    result_type=ExtractedEntities,  # Pydantic validation built in
    system_prompt="Extract named entities from the provided text as JSON.",
)

result = await agent.run("JSON was invented by Douglas Crockford at Yahoo in 2001.")
entities: ExtractedEntities = result.data
# entities.people == ["Douglas Crockford"]
# entities.organizations == ["Yahoo"]

PydanticAI handles the structured output schema generation, validation, and retry loop automatically. You define the Pydantic model; it handles the rest.

Repairing Broken Agent JSON

When validation fails because the JSON itself is malformed (not just wrong values), you need repair before validation:

import requests

def repair_json(broken_json: str) -> dict:
    """Repair malformed JSON using AI JSON Medic API."""
    # For local repair without API calls, use the json-repair library:
    try:
        from json_repair import repair_json as local_repair
        repaired = local_repair(broken_json)
        return json.loads(repaired)
    except Exception:
        pass

    # Fallback: try standard parse
    try:
        return json.loads(broken_json)
    except json.JSONDecodeError:
        raise ValueError(f"Could not repair JSON: {broken_json[:100]}...")

def safe_parse_agent_output(raw: str) -> dict:
    try:
        return json.loads(raw)
    except json.JSONDecodeError:
        return repair_json(raw)

The json-repair Python library (pip install json-repair) handles most common agent output failures: truncation, trailing commas, single quotes, unescaped characters. For complex repair with explanation, AI JSON Medic provides a full 16-stage repair pipeline.

JSON Schema for Agent Tool Definitions

When defining tools for agents, write schemas that guide the model toward valid outputs:

# Vague — model doesn't know what format to use
{"priority": {"type": "string"}}

# Better — explicit enum
{"priority": {"type": "string", "enum": ["low", "medium", "high", "critical"]}}

# Vague — model may output any number
{"confidence": {"type": "number"}}

# Better — constrained range with description
{
  "confidence": {
    "type": "number",
    "minimum": 0.0,
    "maximum": 1.0,
    "description": "Confidence score from 0.0 (no confidence) to 1.0 (certain)"
  }
}

Explicit constraints in the schema reduce the rate of validation failures by giving the model clear guidance during generation. Clear description fields further reduce errors.

Monitoring Agent JSON Validation in Production

Track these metrics to understand your agent's JSON health:

from dataclasses import dataclass, field
from typing import Counter

@dataclass
class AgentValidationMetrics:
    total_outputs: int = 0
    validation_successes: int = 0
    validation_failures: int = 0
    repair_attempts: int = 0
    repair_successes: int = 0
    failure_types: Counter = field(default_factory=Counter)

    def record_success(self):
        self.total_outputs += 1
        self.validation_successes += 1

    def record_failure(self, error_type: str):
        self.total_outputs += 1
        self.validation_failures += 1
        self.failure_types[error_type] += 1

    @property
    def success_rate(self) -> float:
        if self.total_outputs == 0:
            return 0.0
        return self.validation_successes / self.total_outputs

A success rate below 95% indicates your prompts or schemas need refinement. Below 90% means you have a structural problem with your agent design.

Common Agent JSON Validation Failures and Fixes

Missing required fields: The model omitted a field the schema requires. Fix: Add the field to the schema description with a clear note that it's required. Use default in your Pydantic model for fields that can reasonably be omitted. Wrong enum value: The model used "High" instead of "high". Fix: Add case-insensitive validation in Pydantic, or use Literal["high", "High", "HIGH"] and normalize in a validator. Truncated arrays: The model hit a context limit mid-array. Fix: Detect incomplete arrays (last element is truncated), trigger a repair or re-run. The json-repair library handles truncated JSON well. Nested schema drift: After many turns, the model starts generating simplified versions of nested schemas. Fix: Re-inject the full schema definition in the system prompt every N turns. Clear context history if it grows too long. Type coercion needed: Model returns "42" (string) instead of 42 (integer). Fix: Use model_validator with mode="before" in Pydantic to coerce types before validation.

---

Frequently Asked Questions

Should I validate tool call arguments or let the tool handle it?

Validate at both points. Tool-level validation gives you specific, domain-meaningful errors. Agent-level validation gives you early failure before any tool side effect occurs. Use both.

How many retries should I allow for validation failures?

Two retries with the validation error as feedback is the standard. Beyond two retries, the model likely doesn't understand the schema — review the schema design rather than keep retrying.

Does JSON Schema validation slow down my agent?

Pydantic validation is microseconds for typical agent payloads. It's negligible compared to the LLM call latency (seconds). Never skip validation to save time.

What if I'm using streaming output?

Collect the full streamed output before validation. Pydantic cannot validate partial JSON. Use json-repair if the streamed output is truncated before you finish collecting.

Can I use JSON Schema directly without Pydantic?

Yes — use jsonschema (pip install jsonschema) for pure JSON Schema validation without Pydantic models. Useful when your schema comes from an external source (OpenAPI spec, user input) rather than Python model definitions.

How do I debug why an agent keeps failing validation?

Log the raw LLM output before any parsing or validation. Compare it against your schema. The most common cause is a mismatch between the schema you think you're sending and the schema that actually reaches the model. Print both.