Chapter 6: Conversational Memory and `MessagesState`

Up until this point, we have built rock-solid foundations. We understand the State layer, Overwrite semantics, List Reducers, and the structural Topology of nodes, edges, branches, and loops.

Now, we must solve one of the hardest engineering challenges in Generative AI: Conversational Memory.

Every time an LLM is called, it has total amnesia. It has no idea what you said 5 minutes ago. To create the illusion of a continuous conversation, you must manually pass the entire history of the conversation to the LLM on every single request.

In this chapter, we will explore LangGraph's elegant, built-in container for conversation history, trace how lists grow during execution, and build our first conversational graph.

---

6.1 Why Chat History Is Hard

To track conversation history, you need a list of messages.

In Chapter 4, we learned that to safely maintain a list in LangGraph, we must use a Reducer (operator.add). You might be tempted to build conversational memory like this:

# Naive approach
class ChatState(TypedDict):
    messages: Annotated[list[str], operator.add]

This will quickly fail in production for two reasons:

1. Lack of Roles: LLMs need to know who said what. A simple list of strings (["Hello", "Hi there"]) does not differentiate between the User, the AI, or a System instruction.
2. Duplicate Appends: If you run a graph loop 5 times to correct a mistake, a standard operator.add reducer will blindly append the same message 5 times, destroying your context window.

---

6.2 The Message Hierarchy (BaseMessage)

LangChain solves the "Role" problem by wrapping all text inside BaseMessage objects. Instead of passing strings, we pass objects that explicitly identify the author.

HumanMessage

Represents input from the user.

from langchain_core.messages import HumanMessage
msg = HumanMessage(content="What is the weather?")

AIMessage

Represents the response generated by the Large Language Model.

from langchain_core.messages import AIMessage
msg = AIMessage(content="The weather is 72 degrees.")

SystemMessage

Represents the hidden instructions that guide the AI's behavior. The user never sees this.

from langchain_core.messages import SystemMessage
msg = SystemMessage(content="You are a helpful customer support bot.")

ToolMessage

Represents the result returned by an external tool (like a Database or Google Search). This becomes absolutely essential in Chapter 8 when we build Agents.

from langchain_core.messages import ToolMessage
msg = ToolMessage(content="Current temperature in NY is 72", tool_call_id="call_abc123")

---

6.3 Demystifying MessagesState

Because managing a list of BaseMessage objects is required for almost every AI application, LangGraph provides a pre-built state class called MessagesState.

Beginners often treat MessagesState like magic. It is not. It is simply a convenience class built entirely on concepts you already know.

If you were to write MessagesState yourself, it would look exactly like this:

from typing import Annotated, TypedDict
from langgraph.graph.message import add_messages
from langchain_core.messages import AnyMessage

class SimplifiedMessagesState(TypedDict):
    # It is just a list of messages, using a custom reducer called 'add_messages'
    messages: Annotated[list[AnyMessage], add_messages]

The Magic of add_messages (ID Deduplication)

The true power of MessagesState lies in the add_messages reducer function.

When you append a message using add_messages, the Graph Runtime looks at the unique message.id property of the BaseMessage object.

• If the ID is new, it appends the message to the end of the list.
• If the ID already exists in the list, it updates the existing message in place.

Crucial Concept: Deduplication is based strictly on message IDs, not message text. If a node returns two AIMessage("Hello") objects without IDs, they will both be appended.

---

6.4 MessagesState vs Custom ChatState

While you can inherit directly from MessagesState, production applications almost always extend it to include business-specific fields.

from langgraph.graph import MessagesState

# By inheriting, we automatically get the 'messages' key and reducer
class CustomChatState(MessagesState):
    user_id: int
    session_id: str
    active_ticket: str | None

---

6.5 Tracing Message Growth (Without LLMs)

Before we introduce a Large Language Model, let's visualize exactly how MessagesState aggregates history over time.

from langgraph.graph import StateGraph, START, END, MessagesState
from langchain_core.messages import HumanMessage, AIMessage

# A pure Python node that just returns a hardcoded AI message
def mock_bot(state: MessagesState):
    return {"messages": [AIMessage(content="Hello human.")]}

builder = StateGraph(MessagesState)
builder.add_node("bot", mock_bot)
builder.add_edge(START, "bot")
builder.add_edge("bot", END)

app = builder.compile()

# --- Execution Trace ---

print("--- Turn 1 ---")
# The user speaks. We invoke the graph with a HumanMessage.
result_1 = app.invoke({"messages": [HumanMessage(content="Hi")]})

State After Turn 1:

messages
├── HumanMessage("Hi")
└── AIMessage("Hello human.")

print("\n--- Turn 2 ---")
# To continue the conversation, the caller MUST pass the history back in!
result_2 = app.invoke({
    "messages": result_1["messages"] + [HumanMessage(content="What is my name?")]
})

State After Turn 2:

messages
├── HumanMessage("Hi")
├── AIMessage("Hello human.")
├── HumanMessage("What is my name?")
└── AIMessage("Hello human.")

Important Distinction: At this stage, memory is being managed entirely by our application code outside the graph (result_1["messages"] + [...]). MessagesState is simply a container. Later in Chapter 10, we will introduce Checkpointing, where LangGraph itself will automatically persist and load conversational state across server restarts.

---

6.6 Building an LLM Chatbot

Now let's replace our mocked node with a real Large Language Model.

from langchain_openai import ChatOpenAI
from langchain_core.messages import SystemMessage

# Initialize the LLM
llm = ChatOpenAI(model="gpt-4o-mini")

def chat_node(state: MessagesState):
    # Read the existing history from the State container
    chat_history = state["messages"]
    
    # Prepend a SystemMessage to dictate personality
    system_instruction = SystemMessage(content="You are a sarcastic robot.")
    prompt_messages = [system_instruction] + chat_history
    
    # Invoke the LLM
    response = llm.invoke(prompt_messages)
    
    # Return the new AIMessage as a state update. 
    # The 'add_messages' reducer automatically appends this to the history!
    return {"messages": [response]}

# Rebuild the graph using the real LLM node
builder = StateGraph(MessagesState)
builder.add_node("bot", chat_node)
builder.add_edge(START, "bot")
builder.add_edge("bot", END)
real_app = builder.compile()

# Execute
result = real_app.invoke({"messages": [HumanMessage(content="Hi, my name is Krishna.")]})
print(result["messages"][-1].content) # "Oh great, another human. Hello Krishna."

---

6.7 Common Memory Mistakes

As you build conversational graphs, beware of these three fatal pitfalls:

1. Treating MessagesState Like a Database

Beginners often pull thousands of messages from their production Postgres database and inject them directly into MessagesState.

MessagesState represents the active context window, not permanent storage. If you load 5 years of chat history into it, you will crash the LLM immediately.

2. Forgetting to Pass the History Back In

In the example above, if you invoke the graph a second time with just {"messages": [HumanMessage(content="What is my name?")]}, the graph will start completely fresh and have amnesia. Until you implement Checkpointing (Chapter 10), you must manually append the old history array on subsequent API calls.

3. Infinite Context Window Blowouts

Even if you do everything correctly, if a user talks to your chatbot for 3 hours straight, the messages list will eventually exceed the LLM's maximum token limit (e.g., 128k tokens). You must implement a "Trimming" node or a "Summarization" node to safely prune the oldest messages from the list before passing it to the LLM.

---

6.8 Summary

By leveraging the BaseMessage hierarchy and the add_messages reducer, LangGraph makes conversational context incredibly robust.

You no longer have to worry about ID deduplication or accidental overwrites.

Now that we have successfully integrated our first Large Language Model into a node, we must explore exactly what LLMs are capable of. In the next chapter, we will master LLM Nodes, Prompting, and Structured Output.