Chapter 13: Multi-Agent Systems — Mastering LangGraph: From State Machines to Production AI Agents

Welcome to the summit.

If you search the internet for "Multi-Agent Systems" (MAS), you will find endless hype, complex academic papers, and frameworks promising that swarms of AI agents will autonomously run your company. You will see startups raising millions of dollars on the promise of "AI Employees."

However, because you have spent the last 12 chapters mastering LangGraph from the ground up, you possess a secret advantage: You know that "Multi-Agent Systems" are not magic.

A Multi-Agent System is simply:

Specialist Subgraphs (Chapter 12)
LLM Routing Nodes (Chapter 7)
Durable State (Chapter 10)

When you compose these three things together, you create teams of AI agents that can collaborate, debate, and delegate. In this massive chapter, we will explore the architectural patterns used to build enterprise-grade Multi-Agent Systems, the pitfalls you will encounter, and how to structure your state for massive parallel teams.

13.1 The Philosophy of Multi-Agent Systems

Why build a Multi-Agent System at all? If modern LLMs like GPT-4 are so smart, why not just write one massive prompt: "You are an expert researcher, writer, and coder. Do everything."

This is called the God Prompt Anti-Pattern.

When an LLM is asked to do too many things at once, its attention mechanism fails. It forgets instructions. It gets confused about which tools to use. It hallucinates.

By splitting the workload into Multiple Agents, we achieve:

Narrow Focus: A Research Agent only has search tools and a prompt about researching. It literally cannot hallucinate code because it doesn't know how to code.
State Isolation: The Coder Agent doesn't need to read the 50 pages of raw HTML the Research Agent scraped. It only needs the summarized output.
Parallel Execution: Five different research agents can work simultaneously (Chapter 18) before handing their findings to a single Writer Agent.

13.2 The Supervisor Pattern

The most reliable and widely used Multi-Agent architecture in production is the Supervisor Pattern.

Instead of letting a swarm of agents talk to each other chaotically, you introduce a strict hierarchy. You create one central "Supervisor Agent" whose only job is to evaluate the user's request and delegate the work to "Specialist Agents."

               User Request
                    ↓
               [Supervisor]
               ↙    ↓    ↘
      [Searcher] [Writer] [Coder]

The Supervisor does not search the web or write code. It only routes.
The Specialists do not talk to each other. They only talk to the Supervisor.

This strict hub-and-spoke topology is incredibly easy to build in LangGraph and prevents infinite conversational loops.

13.3 Defining the Specialist Agents

A Specialist Agent is simply a LangGraph Node (or an entire Subgraph) that has been given a very narrow prompt and a very specific set of tools.

By keeping the Specialists narrow, you drastically reduce hallucinations.

from langchain_core.messages import AIMessage, SystemMessage

# The Search Specialist (Has access to web tools)
def search_agent_node(state: GlobalState):
    # Narrow persona
    system_prompt = SystemMessage(content="""
    You are a Research Specialist. 
    Your ONLY job is to use your web tools to find accurate facts.
    Do not attempt to write the final article or format the data.
    """)
    
    # ... invoke LLM with tools ...
    
    # Notice we add name="Searcher"?
    # This is critical so the Supervisor knows exactly WHO generated this message.
    return {"messages": [AIMessage(content="Found the facts.", name="Searcher")]}

# The Coder Specialist (Has access to python execution tools)
def coder_agent_node(state: GlobalState):
    system_prompt = SystemMessage(content="""
    You are a Python Specialist. 
    Your ONLY job is to write, execute, and debug Python code.
    """)
    # ... invoke LLM with tools ...
    return {"messages": [AIMessage(content="Code executed successfully.", name="Coder")]}

13.4 The Supervisor Router

The Supervisor is just an LLM Node utilizing .with_structured_output() (which we mastered in Chapter 7) to make a strict routing decision.

from pydantic import BaseModel, Field
from typing import Literal

# 1. Define the exact routing choices using Pydantic
class SupervisorDecision(BaseModel):
    next_agent: Literal["Searcher", "Coder", "FINISH"] = Field(
        description="Who should act next? If the task is fully complete, choose FINISH."
    )
    reasoning: str = Field(
        description="A brief explanation of WHY you chose this agent."
    )

# 2. The Supervisor Node
def supervisor_node(state: GlobalState):
    supervisor_llm = llm.with_structured_output(SupervisorDecision)
    
    # The Prompt Engineering here is critical!
    prompt = f"""You are the Executive Supervisor.
    Review the conversation history and decide who acts next.
    
    Available Agents:
    - Searcher: Use when you need facts from the internet.
    - Coder: Use when you need math or software logic executed.
    
    Conversation History:
    {state['messages']}
    """
    
    decision = supervisor_llm.invoke(prompt)
    
    print(f"Supervisor decided: {decision.next_agent} because {decision.reasoning}")
    
    # We update the state with the string name of the next agent
    return {"next_route": decision.next_agent}

By forcing the Supervisor to output reasoning alongside next_agent (a technique known as Chain-of-Thought prompting), the routing accuracy increases significantly.

13.5 Agent-to-Agent Communication (Shared vs Isolated State)

When the Searcher finds data, how does the Coder know about it?

There are two ways to architect State in a Multi-Agent System:

1. Shared State (The Global Scratchpad)

This is the simplest method. The Supervisor and all Specialists share the exact same GlobalState dictionary. The messages array acts as a public group chat. Every agent can see every message generated by every other agent.

Pros: Easy to build. Context is shared perfectly.
Cons: If the Searcher generates 100 pages of logs, it pollutes the Coder's context window.

2. Isolated State (Subgraphs)

This is the enterprise method (using Chapter 12). The Supervisor holds the ParentState. It delegates to the SearcherSubgraph using an adapter. The Searcher does its work in isolation, and the adapter only returns a polished "final_summary" back to the Supervisor's ParentState.

# Adapter Node for Isolated State
def isolated_searcher_node(state: ParentState):
    # Pass only the exact question to the child
    child_input = {"question": state["current_task"]}
    
    # Invoke the massive research subgraph
    child_result = searcher_subgraph.invoke(child_input)
    
    # Return ONLY the final polished summary to the Parent's Shared Message Array
    summary_message = AIMessage(content=child_result["summary"], name="Searcher")
    
    return {"messages": [summary_message]}

Pros: Prevents context window explosion. Isolates errors.
Cons: Harder to architect. Requires strict State Mapping.

13.6 Building a Complete Supervisor System

Let's build a functional Supervisor Graph using the Shared State method to see the topology.

from langgraph.graph import StateGraph, START, END, MessagesState

# 1. Shared State
class TeamState(MessagesState):
    next_route: str 

# 2. The Nodes (Implementations hidden for brevity)
# def searcher_node(state) ...
# def writer_node(state) ...
# def supervisor_node(state) ...

# 3. Routing Edge Function
def router_edge(state: TeamState):
    # Just return the string the Supervisor decided on!
    if state["next_route"] == "FINISH":
        return END
    return state["next_route"]

# 4. Build Topology
builder = StateGraph(TeamState)

builder.add_node("supervisor", supervisor_node)
builder.add_node("searcher", searcher_node)
builder.add_node("writer", writer_node)

# Execution always starts at the Supervisor
builder.add_edge(START, "supervisor")

# Supervisor branches out to the Specialists or END
builder.add_conditional_edges("supervisor", router_edge)

# When a Specialist finishes, it MUST route back to the Supervisor!
builder.add_edge("searcher", "supervisor")
builder.add_edge("writer", "supervisor")

app = builder.compile()

Execution Trace:

STARTING MULTI-AGENT SYSTEM

[Supervisor] Evaluating state...
Supervisor decided: searcher because I need to find the founding date of Python.
-> Searcher is working...

[Supervisor] Evaluating state...
Supervisor decided: writer because the facts have been gathered.
-> Writer is working...

[Supervisor] Evaluating state...
Supervisor decided: FINISH because the article is complete.

13.7 Hierarchical Teams

What if your project is so large that a single Supervisor gets confused? A Supervisor managing 15 different agents will suffer from the same attention degradation as the God Prompt.

Because LangGraph allows graphs to become nodes (Chapter 12), you can build Hierarchical Teams.

You can build a "Marketing Supervisor" that manages Writers and SEO agents. You can build an "Engineering Supervisor" that manages Coders and Testers. Then, you build a "CEO Agent" (a Parent Graph) whose only job is to route tasks between the Marketing Supervisor and the Engineering Supervisor.

                     [CEO Agent]
                    /           \
        [Marketing Sup]       [Engineering Sup]
          /         \           /           \
     [Writer]     [SEO]     [Coder]       [Tester]

This is how massive enterprise software factories are built using LLMs.

13.8 Swarm Architectures (Peer-to-Peer)

The Supervisor Pattern relies on a central hub. However, there is another pattern called Swarm Architecture (or Network Architecture).

In a Swarm, there is no Supervisor. Instead, every agent is allowed to route directly to every other agent.

[Sales Agent] ↔ [Support Agent] ↔ [Billing Agent]

If the User asks the Sales Agent for a refund, the Sales Agent invokes a structured output routing decision: "I can't do this, I am transferring the user to the Billing Agent." The edge routes directly from Sales to Billing.

Pros: Highly fluid, mimics human corporate structures (handoffs).
Cons: Extremely difficult to debug. Infinite loops are rampant because agents can bounce the user back and forth forever. It requires meticulous prompt engineering.

13.9 Common Anti-Patterns

When building Multi-Agent Systems, watch out for these traps:

The Echo Chamber: Two agents are instructed to debate. Agent A says X. Agent B agrees with X. Agent A thanks Agent B. Agent B thanks Agent A. They loop endlessly. Always use MAX_ITERATIONS failsafes (Chapter 9).
The Delegation Loop: The Supervisor delegates to the Coder. The Coder fails to write the code, apologizes, and returns control to the Supervisor. The Supervisor reads the apology, and immediately delegates to the Coder again. This happens when the Coder isn't given strict instructions on how to handle failures.
Identity Confusion: If you forget to attach name="AgentName" to your AIMessage objects, the Supervisor reads the state array and cannot figure out who actually performed the work.

13.10 Summary

A Multi-Agent System is not a mysterious new black box. It is simply the beautiful orchestration of LangGraph Topologies, LLM Structured Output routing, and Subgraph isolation.

By mastering the Supervisor pattern, you can break massive, impossible prompts into tiny, highly reliable Specialist Agents.

As our applications grow in size and complexity, waiting 45 seconds for a massive multi-agent system to finish before returning an answer to the user becomes unacceptable.

In the next chapter, we will solve the user experience problem by mastering Streaming and Real-Time Updates.