Chapter 15: Memory Beyond Messages — Mastering LangGraph: From State Machines to Production AI Agents

In Chapter 10, we mastered Checkpointing. By saving our MessagesState to a Postgres database under a specific thread_id, we achieved persistent conversational memory.

However, a thread_id represents a single conversation.

If a user talks to your bot today about their dog, and then starts a brand new conversation (thread_id = 2) tomorrow, the bot will have total amnesia about the dog. To the user, this feels disjointed and frustrating.

In this chapter, we will explore Long-Term Memory. We will learn how to extract facts from short-term conversations, store them permanently, and retrieve them across completely different threads to build highly personalized AI experiences.

15.1 Short-Term vs Long-Term Memory

Before writing code, we must establish a rigid architectural distinction:

Short-Term Memory (Thread State): The exact, turn-by-turn history of the current conversation. It is stored in MessagesState. It is extremely detailed, but mathematically bounded. If you let a single thread run for 6 months, the MessagesState array will eventually exceed the LLM's 128k token context window, causing the application to crash.
Long-Term Memory (Semantic/Profile Memory): Extracted, summarized facts about the user ("User lives in NY", "User owns a dog"). It is stored in a separate database (often a Vector DB or a traditional SQL table) and injected selectively into new conversations. It is unbounded.

15.2 Managing User Profiles (The SQL Approach)

The simplest form of Long-Term Memory is a structured User Profile.

Instead of forcing the LLM to read through 500 old chat messages to remember the user's name and occupation, we actively extract that data and save it to a database table.

You can build an "Extraction Node" that runs in parallel or at the end of your graph to update the user's profile.

from pydantic import BaseModel
import sqlite3 # Using sqlite for demonstration

class ProfileUpdate(BaseModel):
    user_name: str | None
    user_city: str | None
    user_occupation: str | None
    
def extract_profile_node(state: MessagesState):
    # Use Structured Output to hunt for profile facts in the latest message
    extractor = llm.with_structured_output(ProfileUpdate)
    
    # We only analyze the most recent message to save tokens
    latest_msg = state["messages"][-1].content
    new_facts = extractor.invoke(f"Extract user details if present: {latest_msg}")
    
    # Save to your external database
    conn = sqlite3.connect("users.db")
    cursor = conn.cursor()
    
    if new_facts.user_name:
        cursor.execute("UPDATE profiles SET name = ? WHERE user_id = ?", 
                      (new_facts.user_name, state["user_id"]))
    # ... update other fields ...
    
    conn.commit()
    return {} # No internal graph state update needed!

When the user starts a new thread tomorrow, you query the SQL database, fetch their profile, and inject it into the SystemMessage.

def entry_node(state: MessagesState):
    # Fetch profile from SQL
    conn = sqlite3.connect("users.db")
    cursor = conn.cursor()
    cursor.execute("SELECT name, city FROM profiles WHERE user_id = ?", (state["user_id"],))
    row = cursor.fetchone()
    
    system_prompt = f"The user's name is {row[0]} and they live in {row[1]}."
    
    # Inject the long-term memory into the short-term state!
    return {"messages": [SystemMessage(content=system_prompt)]}

15.3 Semantic Memory and Vector Databases

Extracting discrete fields (like Name and City) is easy. But what if the user casually mentions: "I really hated the ending of Game of Thrones, the pacing was terrible."

You cannot build a SQL column for every possible abstract fact. For unstructured facts, we use Semantic Memory powered by Embeddings and Vector Databases (like Pinecone, Milvus, Qdrant, or pgvector).

The Ingestion Process

Memory Generation Node: An LLM node summarizes the current conversation and extracts a list of core "Memories" (e.g., "User dislikes Game of Thrones").
Embedding: The text is converted into an embedding array (a numerical representation of meaning) using a model like OpenAI's text-embedding-3-small.
Storage: The array is saved to the Vector DB alongside metadata (user_id, timestamp).

from langchain_openai import OpenAIEmbeddings
from langchain_community.vectorstores import Pinecone

embeddings = OpenAIEmbeddings(model="text-embedding-3-small")
# Assume vector_store is initialized

def save_memory_node(state: MessagesState):
    # Ask LLM to extract abstract facts
    memory_extractor = llm.invoke(
        f"Extract core preferences or facts from this chat: {state['messages']}"
    )
    
    fact_text = memory_extractor.content
    
    # Convert to numbers and save to Pinecone!
    vector_store.add_texts(
        texts=[fact_text],
        metadatas=[{"user_id": state["user_id"], "type": "preference"}]
    )
    return {}

15.4 Memory Retrieval Nodes

When a user starts a new conversation, how do we get those semantic memories back?

We build a Memory Retrieval Node at the very beginning of our graph topology.

START ➔ [Memory Retrieval] ➔ [Agent] ➔ END

def memory_retrieval_node(state: MessagesState):
    user_query = state["messages"][-1].content
    
    # 1. Search the Vector Database for similar past memories!
    # We filter strictly by the user_id to avoid data leaks
    past_memories = vector_store.similarity_search(
        query=user_query, 
        k=3, # get top 3 memories
        filter={"user_id": state["user_id"]}
    )
    
    # 2. Format the retrieved memories into context
    memory_context = "\n".join([f"- {doc.page_content}" for doc in past_memories])
    
    system_instruction = SystemMessage(
        content=f"""Use these past memories to personalize your response:
        <memories>
        {memory_context}
        </memories>
        """
    )
    
    # 3. Prepend the injected memories to the state
    return {"messages": [system_instruction]}

If the user asks: "What should I watch tonight?", the memory_retrieval_node will automatically find the vector representing "User dislikes Game of Thrones", inject it into the prompt, and the Agent will intelligently avoid recommending similar shows.

15.5 Managing Memory Decay (Forgetting)

One of the biggest flaws in Long-Term Memory systems is that users change.

If a user states "I am a vegetarian" in 2024, the system commits that to Semantic Memory. In 2026, the user states "I eat steak now."

If your Vector DB retrieves both memories and injects them into the prompt, the LLM will hallucinate or get confused.

Enterprise systems implement Memory Decay. When saving memories to the Vector DB, you always attach a timestamp metadata field. When retrieving memories, you either filter out memories older than 12 months, or you pass the timestamps to the LLM and add a prompt instruction: "If memories conflict, trust the most recent timestamp."

15.6 Summary

By separating Short-Term MessagesState from Long-Term Database storage, you can build AI applications that "remember" users perfectly across years of interaction without ever exceeding the context window.

This process of searching a Vector Database and injecting facts into the LLM's prompt is a foundational AI architecture.

In the next chapter, we will dedicate an entire deep dive to the enterprise manifestation of this pattern, used to ingest thousands of private documents rather than just user preferences: Retrieval Augmented Generation (RAG).