{
 "cells": [
  {
   "cell_type": "markdown",
   "id": "2c7bdc91-9b89-4e59-bc27-89508b024635",
   "metadata": {},
   "source": [
    "# Migrating from MapReduceDocumentsChain\n",
    "\n",
    "[MapReduceDocumentsChain](https://python.langchain.com/api_reference/langchain/chains/langchain.chains.combine_documents.map_reduce.MapReduceDocumentsChain.html) implements a map-reduce strategy over (potentially long) texts. The strategy is as follows:\n",
    "\n",
    "- Split a text into smaller documents;\n",
    "- Map a process onto the smaller documents;\n",
    "- Reduce or consolidate the results of the process into a final result.\n",
    "\n",
    "Note that the map step is typically parallelized over the input documents.\n",
    "\n",
    "A common process applied in this context is summarization, in which the map step summarizes individual documents, and the reduce step generates a summary of the summaries.\n",
    "\n",
    "In the reduce step, `MapReduceDocumentsChain` supports a recursive \"collapsing\" of the summaries: the inputs would be partitioned based on a token limit, and summaries would be generated of the partitions. This step would be repeated until the total length of the summaries was within a desired limit, allowing for the summarization of arbitrary-length text. This is particularly useful for models with smaller context windows.\n",
    "\n",
    "LangGraph suports [map-reduce](https://langchain-ai.github.io/langgraph/how-tos/map-reduce/) workflows, and confers a number of advantages for this problem:\n",
    "\n",
    "- LangGraph allows for individual steps (such as successive summarizations) to be streamed, allowing for greater control of execution;\n",
    "- LangGraph's [checkpointing](https://langchain-ai.github.io/langgraph/how-tos/persistence/) supports error recovery, extending with human-in-the-loop workflows, and easier incorporation into conversational applications.\n",
    "- The LangGraph implementation is easier to extend, as we will see below.\n",
    "\n",
    "Below we will go through both `MapReduceDocumentsChain` and a corresponding LangGraph implementation, first on a simple example for illustrative purposes, and second on a longer example text to demonstrate the recursive reduce step.\n",
    "\n",
    "Let's first load a chat model:\n",
    "\n",
    "import ChatModelTabs from \"@theme/ChatModelTabs\";\n",
    "\n",
    "<ChatModelTabs customVarName=\"llm\" />"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 1,
   "id": "0bdf886b-aeeb-407e-81b8-28bad59ad57a",
   "metadata": {},
   "outputs": [],
   "source": [
    "# | output: false\n",
    "# | echo: false\n",
    "\n",
    "from langchain_openai import ChatOpenAI\n",
    "\n",
    "llm = ChatOpenAI(model=\"gpt-4o-mini\", temperature=0)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "41cfb569-f7e6-48cb-a90e-45a482009971",
   "metadata": {},
   "source": [
    "## Basic example (short documents)\n",
    "\n",
    "Let's use the following 3 documents for illustrative purposes."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 2,
   "id": "b221a71f-982b-4c08-8597-96c890e00965",
   "metadata": {},
   "outputs": [],
   "source": [
    "from langchain_core.documents import Document\n",
    "\n",
    "documents = [\n",
    "    Document(page_content=\"Apples are red\", metadata={\"title\": \"apple_book\"}),\n",
    "    Document(page_content=\"Blueberries are blue\", metadata={\"title\": \"blueberry_book\"}),\n",
    "    Document(page_content=\"Bananas are yelow\", metadata={\"title\": \"banana_book\"}),\n",
    "]"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "c717514a-1b6d-4a0f-9093-ef594b9a0b17",
   "metadata": {},
   "source": [
    "### Legacy\n",
    "\n",
    "<details open>\n",
    "    \n",
    "Below we show an implementation with `MapReduceDocumentsChain`. We define the prompt templates for the map and reduce steps, instantiate separate chains for these steps, and finally instantiate the `MapReduceDocumentsChain`:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 4,
   "id": "84ee3851-b4a9-4fbe-a78f-d05168715b91",
   "metadata": {},
   "outputs": [],
   "source": [
    "from langchain.chains import MapReduceDocumentsChain, ReduceDocumentsChain\n",
    "from langchain.chains.combine_documents.stuff import StuffDocumentsChain\n",
    "from langchain.chains.llm import LLMChain\n",
    "from langchain_core.prompts import ChatPromptTemplate\n",
    "from langchain_text_splitters import CharacterTextSplitter\n",
    "\n",
    "# Map\n",
    "map_template = \"Write a concise summary of the following: {docs}.\"\n",
    "map_prompt = ChatPromptTemplate([(\"human\", map_template)])\n",
    "map_chain = LLMChain(llm=llm, prompt=map_prompt)\n",
    "\n",
    "\n",
    "# Reduce\n",
    "reduce_template = \"\"\"\n",
    "The following is a set of summaries:\n",
    "{docs}\n",
    "Take these and distill it into a final, consolidated summary\n",
    "of the main themes.\n",
    "\"\"\"\n",
    "reduce_prompt = ChatPromptTemplate([(\"human\", reduce_template)])\n",
    "reduce_chain = LLMChain(llm=llm, prompt=reduce_prompt)\n",
    "\n",
    "\n",
    "# Takes a list of documents, combines them into a single string, and passes this to an LLMChain\n",
    "combine_documents_chain = StuffDocumentsChain(\n",
    "    llm_chain=reduce_chain, document_variable_name=\"docs\"\n",
    ")\n",
    "\n",
    "# Combines and iteratively reduces the mapped documents\n",
    "reduce_documents_chain = ReduceDocumentsChain(\n",
    "    # This is final chain that is called.\n",
    "    combine_documents_chain=combine_documents_chain,\n",
    "    # If documents exceed context for `StuffDocumentsChain`\n",
    "    collapse_documents_chain=combine_documents_chain,\n",
    "    # The maximum number of tokens to group documents into.\n",
    "    token_max=1000,\n",
    ")\n",
    "\n",
    "# Combining documents by mapping a chain over them, then combining results\n",
    "map_reduce_chain = MapReduceDocumentsChain(\n",
    "    # Map chain\n",
    "    llm_chain=map_chain,\n",
    "    # Reduce chain\n",
    "    reduce_documents_chain=reduce_documents_chain,\n",
    "    # The variable name in the llm_chain to put the documents in\n",
    "    document_variable_name=\"docs\",\n",
    "    # Return the results of the map steps in the output\n",
    "    return_intermediate_steps=False,\n",
    ")"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 5,
   "id": "4f57ed52-08a5-49f6-ab19-1be51a853a2f",
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Fruits come in a variety of colors, with apples being red, blueberries being blue, and bananas being yellow.\n"
     ]
    }
   ],
   "source": [
    "result = map_reduce_chain.invoke(documents)\n",
    "\n",
    "print(result[\"output_text\"])"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "46d29559-5948-4ce9-b7c5-fa6729cf2485",
   "metadata": {},
   "source": [
    "In the [LangSmith trace](https://smith.langchain.com/public/8d88a2c0-5d26-41f6-9176-d06549b17aa6/r) we observe four LLM calls: one summarizing each of the three input documents, and one summarizing the summaries."
   ]
  },
  {
   "cell_type": "markdown",
   "id": "b5399533-8662-4fad-b885-e3df3d809c44",
   "metadata": {},


{
 "cells": [
  {
   "cell_type": "markdown",
   "id": "9db5ad7a-857e-46ea-9d0c-ba3fbe62fc81",
   "metadata": {},
   "source": [
    "# Migrating from MapRerankDocumentsChain\n",
    "\n",
    "[MapRerankDocumentsChain](https://python.langchain.com/api_reference/langchain/chains/langchain.chains.combine_documents.map_rerank.MapRerankDocumentsChain.html) implements a strategy for analyzing long texts. The strategy is as follows:\n",
    "\n",
    "- Split a text into smaller documents;\n",
    "- Map a process to the set of documents, where the process includes generating a score;\n",
    "- Rank the results by score and return the maximum.\n",
    "\n",
    "A common process in this scenario is question-answering using pieces of context from a document. Forcing the model to generate score along with its answer helps to select for answers generated only by relevant context.\n",
    "\n",
    "An [LangGraph](https://langchain-ai.github.io/langgraph/) implementation allows for the incorporation of [tool calling](/docs/concepts/#functiontool-calling) and other features for this problem. Below we will go through both `MapRerankDocumentsChain` and a corresponding LangGraph implementation on a simple example for illustrative purposes."
   ]
  },
  {
   "cell_type": "markdown",
   "id": "39f11f9f-ac24-485e-bc15-285bebb9c12e",
   "metadata": {},
   "source": [
    "## Example\n",
    "\n",
    "Let's go through an example where we analyze a set of documents. Let's use the following 3 documents:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 2,
   "id": "ef975d40-6ea3-4280-84cb-fae4c285c72b",
   "metadata": {},
   "outputs": [],
   "source": [
    "from langchain_core.documents import Document\n",
    "\n",
    "documents = [\n",
    "    Document(page_content=\"Alice has blue eyes\", metadata={\"title\": \"book_chapter_2\"}),\n",
    "    Document(page_content=\"Bob has brown eyes\", metadata={\"title\": \"book_chapter_1\"}),\n",
    "    Document(\n",
    "        page_content=\"Charlie has green eyes\", metadata={\"title\": \"book_chapter_3\"}\n",
    "    ),\n",
    "]"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "e3b99cfc-b99c-4da8-9c87-903e0249d227",
   "metadata": {},
   "source": [
    "### Legacy\n",
    "\n",
    "<details open>\n",
    "\n",
    "Below we show an implementation with `MapRerankDocumentsChain`. We define the prompt template for a question-answering task and instantiate a [LLMChain](https://python.langchain.com/api_reference/langchain/chains/langchain.chains.llm.LLMChain.html) object for this purpose. We define how documents are formatted into the prompt and ensure consistency among the keys in the various prompts."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 4,
   "id": "3b65e056-d739-4985-8bfc-0edf783f2b16",
   "metadata": {},
   "outputs": [],
   "source": [
    "from langchain.chains import LLMChain, MapRerankDocumentsChain\n",
    "from langchain.output_parsers.regex import RegexParser\n",
    "from langchain_core.prompts import PromptTemplate\n",
    "from langchain_openai import OpenAI\n",
    "\n",
    "document_variable_name = \"context\"\n",
    "llm = OpenAI()\n",
    "# The prompt here should take as an input variable the\n",
    "# `document_variable_name`\n",
    "# The actual prompt will need to be a lot more complex, this is just\n",
    "# an example.\n",
    "prompt_template = (\n",
    "    \"What color are Bob's eyes? \"\n",
    "    \"Output both your answer and a score (1-10) of how confident \"\n",
    "    \"you are in the format: <Answer>\\nScore: <Score>.\\n\\n\"\n",
    "    \"Provide no other commentary.\\n\\n\"\n",
    "    \"Context: {context}\"\n",
    ")\n",
    "output_parser = RegexParser(\n",
    "    regex=r\"(.*?)\\nScore: (.*)\",\n",
    "    output_keys=[\"answer\", \"score\"],\n",
    ")\n",
    "prompt = PromptTemplate(\n",
    "    template=prompt_template,\n",
    "    input_variables=[\"context\"],\n",
    "    output_parser=output_parser,\n",
    ")\n",
    "llm_chain = LLMChain(llm=llm, prompt=prompt)\n",
    "chain = MapRerankDocumentsChain(\n",
    "    llm_chain=llm_chain,\n",
    "    document_variable_name=document_variable_name,\n",
    "    rank_key=\"score\",\n",
    "    answer_key=\"answer\",\n",
    ")"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 9,
   "id": "fe94c2e5-4c56-4604-a16c-055c196f4a57",
   "metadata": {},
   "outputs": [
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "/langchain/libs/langchain/langchain/chains/llm.py:369: UserWarning: The apply_and_parse method is deprecated, instead pass an output parser directly to LLMChain.\n",
      "  warnings.warn(\n"
     ]
    },
    {
     "data": {
      "text/plain": [
       "'Brown'"
      ]
     },
     "execution_count": 9,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "response = chain.invoke(documents)\n",
    "response[\"output_text\"]"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "317e51c2-810f-463b-9da2-604fe95a8b48",
   "metadata": {},
   "source": [
    "Inspecting the [LangSmith trace](https://smith.langchain.com/public/7a071bd1-0283-4b90-898c-6e4a2b5a0593/r) for the above run, we can see three LLM calls-- one for each document-- and that the scoring mechanism mitigated against hallucinations.\n",
    "\n",
    "</details>\n",
    "\n",
    "### LangGraph\n",
    "\n",
    "<details open>\n",
    "\n",
    "Below we show a LangGraph implementation of this process. Note that our template is simplified, as we delegate the formatting instructions to the chat model's tool-calling features via the [.with_structured_output](/docs/how_to/structured_output/) method.\n",
    "\n",
    "Here we follow a basic [map-reduce](https://langchain-ai.github.io/langgraph/how-tos/map-reduce/) workflow to execute the LLM calls in parallel.\n",
    "\n",
    "We will need to install `langgraph`:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "b8fab4f6-eed1-4662-8d3f-82846a2edfb3",
   "metadata": {},
   "outputs": [],
   "source": [
    "pip install -qU langgraph"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 6,
   "id": "b8493533-7ab3-4f75-aab1-390340bff2ea",
   "metadata": {},
   "outputs": [],
   "source": [
    "import operator\n",
    "from typing import Annotated, List, TypedDict\n",
    "\n",
    "from langchain_core.prompts import ChatPromptTemplate\n",
    "from langchain_openai import ChatOpenAI\n",
    "from langgraph.constants import Send\n",
    "from langgraph.graph import END, START, StateGraph\n",
    "\n",
    "\n",
    "class AnswerWithScore(TypedDict):\n",
    "    answer: str\n",
    "    score: Annotated[int, ..., \"Score from 1-10.\"]\n",
    "\n",
    "\n",
    "llm = ChatOpenAI(model=\"gpt-4o-mini\", temperature=0)\n",
    "\n",


[`LLMChain`](https://python.langchain.com/api_reference/langchain/chains/langchain.chains.llm.LLMChain.html) combined a prompt template, LLM, and output parser into a class.

Some advantages of switching to the LCEL implementation are:

- Clarity around contents and parameters. The legacy `LLMChain` contains a default output parser and other options.

- Easier streaming. `LLMChain` only supports streaming via callbacks.

- Easier access to raw message outputs if desired. `LLMChain` only exposes these via a parameter or via callback.

%pip install --upgrade --quiet langchain-openai

'text': "Why couldn't the bicycle stand up by itself?\n\nBecause it was two tired!"}

from langchain_core.prompts import ChatPromptTemplate

[("user", "Tell me a {adjective} joke")],

legacy_chain = LLMChain(llm=ChatOpenAI(), prompt=prompt)

legacy_result = legacy_chain({"adjective": "funny"})

Note that `LLMChain` by default returned a `dict` containing both the input and the output from `StrOutputParser`, so to extract the output, you need to access the `"text"` key.

"Why couldn't the bicycle stand up by itself?\n\nBecause it was two tired!"

'Why was the math book sad?\n\nBecause it had too many problems.'

from langchain_core.output_parsers import StrOutputParser

chain = prompt | ChatOpenAI() | StrOutputParser()

If you'd like to mimic the `dict` packaging of input and output in `LLMChain`, you can use a [`RunnablePassthrough.assign`](https://python.langchain.com/api_reference/core/runnables/langchain_core.runnables.passthrough.RunnablePassthrough.html) like:

'text': 'Why did the scarecrow win an award? Because he was outstanding in his field!'}

from langchain_core.runnables import RunnablePassthrough

See [this tutorial](/docs/tutorials/llm_chain) for more detail on building with prompt templates, LLMs, and output parsers.

Check out the [LCEL conceptual docs](/docs/concepts/#langchain-expression-language-lcel) for more background information.


{
 "cells": [
  {
   "cell_type": "markdown",
   "id": "d20aeaad-b3ca-4a7d-b02d-3267503965af",
   "metadata": {},
   "source": [
    "# Migrating from ConversationalChain\n",
    "\n",
    "[`ConversationChain`](https://python.langchain.com/api_reference/langchain/chains/langchain.chains.conversation.base.ConversationChain.html) incorporated a memory of previous messages to sustain a stateful conversation.\n",
    "\n",
    "Some advantages of switching to the Langgraph implementation are:\n",
    "\n",
    "- Innate support for threads/separate sessions. To make this work with `ConversationChain`, you'd need to instantiate a separate memory class outside the chain.\n",
    "- More explicit parameters. `ConversationChain` contains a hidden default prompt, which can cause confusion.\n",
    "- Streaming support. `ConversationChain` only supports streaming via callbacks.\n",
    "\n",
    "Langgraph's [checkpointing](https://langchain-ai.github.io/langgraph/how-tos/persistence/) system supports multiple threads or sessions, which can be specified via the `\"thread_id\"` key in its configuration parameters."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "b99b47ec",
   "metadata": {},
   "outputs": [],
   "source": [
    "%pip install --upgrade --quiet langchain langchain-openai"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 2,
   "id": "717c8673",
   "metadata": {},
   "outputs": [],
   "source": [
    "import os\n",
    "from getpass import getpass\n",
    "\n",
    "if \"OPENAI_API_KEY\" not in os.environ:\n",
    "    os.environ[\"OPENAI_API_KEY\"] = getpass()"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "00df631d-5121-4918-94aa-b88acce9b769",
   "metadata": {},
   "source": [
    "## Legacy\n",
    "\n",
    "<details open>"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 2,
   "id": "4f2cc6dc-d70a-4c13-9258-452f14290da6",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "{'input': \"I'm Bob, how are you?\",\n",
       " 'history': '',\n",
       " 'response': \"Arrr matey, I be a pirate sailin' the high seas. What be yer business with me?\"}"
      ]
     },
     "execution_count": 2,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "from langchain.chains import ConversationChain\n",
    "from langchain.memory import ConversationBufferMemory\n",
    "from langchain_core.prompts import ChatPromptTemplate\n",
    "from langchain_openai import ChatOpenAI\n",
    "\n",
    "template = \"\"\"\n",
    "You are a pirate. Answer the following questions as best you can.\n",
    "Chat history: {history}\n",
    "Question: {input}\n",
    "\"\"\"\n",
    "\n",
    "prompt = ChatPromptTemplate.from_template(template)\n",
    "\n",
    "memory = ConversationBufferMemory()\n",
    "\n",
    "chain = ConversationChain(\n",
    "    llm=ChatOpenAI(),\n",
    "    memory=memory,\n",
    "    prompt=prompt,\n",
    ")\n",
    "\n",
    "chain({\"input\": \"I'm Bob, how are you?\"})"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 3,
   "id": "53f2c723-178f-470a-8147-54e7cb982211",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "{'input': 'What is my name?',\n",
       " 'history': \"Human: I'm Bob, how are you?\\nAI: Arrr matey, I be a pirate sailin' the high seas. What be yer business with me?\",\n",
       " 'response': 'Your name be Bob, matey.'}"
      ]
     },
     "execution_count": 3,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "chain({\"input\": \"What is my name?\"})"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "f8e36b0e-c7dc-4130-a51b-189d4b756c7f",
   "metadata": {},
   "source": [
    "</details>\n",
    "\n",
    "## Langgraph\n",
    "\n",
    "<details open>"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 4,
   "id": "a59b910c-0d02-41aa-bc99-441f11989cf8",
   "metadata": {},
   "outputs": [],
   "source": [
    "import uuid\n",
    "\n",
    "from langchain_openai import ChatOpenAI\n",
    "from langgraph.checkpoint.memory import MemorySaver\n",
    "from langgraph.graph import START, MessagesState, StateGraph\n",
    "\n",
    "model = ChatOpenAI(model=\"gpt-4o-mini\")\n",
    "\n",
    "# Define a new graph\n",
    "workflow = StateGraph(state_schema=MessagesState)\n",
    "\n",
    "\n",
    "# Define the function that calls the model\n",
    "def call_model(state: MessagesState):\n",
    "    response = model.invoke(state[\"messages\"])\n",
    "    return {\"messages\": response}\n",
    "\n",
    "\n",
    "# Define the two nodes we will cycle between\n",
    "workflow.add_edge(START, \"model\")\n",
    "workflow.add_node(\"model\", call_model)\n",
    "\n",
    "# Add memory\n",
    "memory = MemorySaver()\n",
    "app = workflow.compile(checkpointer=memory)\n",
    "\n",
    "\n",
    "# The thread id is a unique key that identifies\n",
    "# this particular conversation.\n",
    "# We'll just generate a random uuid here.\n",
    "thread_id = uuid.uuid4()\n",
    "config = {\"configurable\": {\"thread_id\": thread_id}}"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 5,
   "id": "3a9df4bb-e804-4373-9a15-a29dc0371595",
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "================================\u001b[1m Human Message \u001b[0m=================================\n",
      "\n",
      "I'm Bob, how are you?\n",
      "==================================\u001b[1m Ai Message \u001b[0m==================================\n",
      "\n",
      "Ahoy, Bob! I be feelin' as lively as a ship in full sail! How be ye on this fine day?\n"
     ]
    }
   ],
   "source": [
    "query = \"I'm Bob, how are you?\"\n",
    "\n",
    "input_messages = [\n",
    "    {\n",
    "        \"role\": \"system\",\n",
    "        \"content\": \"You are a pirate. Answer the following questions as best you can.\",\n",
    "    },\n",
    "    {\"role\": \"user\", \"content\": query},\n",
    "]\n",
    "for event in app.stream({\"messages\": input_messages}, config, stream_mode=\"values\"):\n",
    "    event[\"messages\"][-1].pretty_print()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 6,
   "id": "d3f77e69-fa3d-496c-968c-86371e1e8cf1",
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "================================\u001b[1m Human Message \u001b[0m=================================\n",
      "\n",
      "What is my name?\n",
      "==================================\u001b[1m Ai Message \u001b[0m==================================\n",
      "\n",


{
 "cells": [
  {
   "cell_type": "markdown",
   "id": "292a3c83-44a9-4426-bbec-f1a778d00d93",
   "metadata": {},
   "source": [
    "# Migrating from ConversationalRetrievalChain\n",
    "\n",
    "The [`ConversationalRetrievalChain`](https://python.langchain.com/api_reference/langchain/chains/langchain.chains.conversational_retrieval.base.ConversationalRetrievalChain.html) was an all-in one way that combined retrieval-augmented generation with chat history, allowing you to \"chat with\" your documents.\n",
    "\n",
    "Advantages of switching to the LCEL implementation are similar to the [`RetrievalQA` migration guide](./retrieval_qa.ipynb):\n",
    "\n",
    "- Clearer internals. The `ConversationalRetrievalChain` chain hides an entire question rephrasing step which dereferences the initial query against the chat history.\n",
    "  - This means the class contains two sets of configurable prompts, LLMs, etc.\n",
    "- More easily return source documents.\n",
    "- Support for runnable methods like streaming and async operations.\n",
    "\n",
    "Here are equivalent implementations with custom prompts.\n",
    "We'll use the following ingestion code to load a [blog post by Lilian Weng](https://lilianweng.github.io/posts/2023-06-23-agent/) on autonomous agents into a local vector store:\n",
    "\n",
    "## Shared setup\n",
    "\n",
    "For both versions, we'll need to load the data with the `WebBaseLoader` document loader, split it with `RecursiveCharacterTextSplitter`, and add it to an in-memory `FAISS` vector store.\n",
    "\n",
    "We will also instantiate a chat model to use."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "b99b47ec",
   "metadata": {},
   "outputs": [],
   "source": [
    "%pip install --upgrade --quiet langchain-community langchain langchain-openai faiss-cpu beautifulsoup4"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 2,
   "id": "717c8673",
   "metadata": {},
   "outputs": [],
   "source": [
    "import os\n",
    "from getpass import getpass\n",
    "\n",
    "if \"OPENAI_API_KEY\" not in os.environ:\n",
    "    os.environ[\"OPENAI_API_KEY\"] = getpass()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 2,
   "id": "44119498-5a98-4077-9e2f-c75500e7eace",
   "metadata": {},
   "outputs": [],
   "source": [
    "# Load docs\n",
    "from langchain_community.document_loaders import WebBaseLoader\n",
    "from langchain_community.vectorstores import FAISS\n",
    "from langchain_openai.chat_models import ChatOpenAI\n",
    "from langchain_openai.embeddings import OpenAIEmbeddings\n",
    "from langchain_text_splitters import RecursiveCharacterTextSplitter\n",
    "\n",
    "loader = WebBaseLoader(\"https://lilianweng.github.io/posts/2023-06-23-agent/\")\n",
    "data = loader.load()\n",
    "\n",
    "# Split\n",
    "text_splitter = RecursiveCharacterTextSplitter(chunk_size=500, chunk_overlap=0)\n",
    "all_splits = text_splitter.split_documents(data)\n",
    "\n",
    "# Store splits\n",
    "vectorstore = FAISS.from_documents(documents=all_splits, embedding=OpenAIEmbeddings())\n",
    "\n",
    "# LLM\n",
    "llm = ChatOpenAI()"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "8bc06416",
   "metadata": {},
   "source": [
    "## Legacy\n",
    "\n",
    "<details open>"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 5,
   "id": "8b471e7d-3ccb-4ab3-bc09-304c4b14a908",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "{'question': 'What are autonomous agents?',\n",
       " 'chat_history': '',\n",
       " 'answer': 'Autonomous agents are entities empowered with capabilities like planning, task decomposition, and memory to perform complex tasks independently. These agents can leverage tools like browsing the internet, reading documentation, executing code, and calling APIs to achieve their objectives. They are designed to handle tasks like scientific discovery and experimentation autonomously.'}"
      ]
     },
     "execution_count": 5,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "from langchain.chains import ConversationalRetrievalChain\n",
    "from langchain_core.prompts import ChatPromptTemplate\n",
    "\n",
    "condense_question_template = \"\"\"\n",
    "Given the following conversation and a follow up question, rephrase the follow up question to be a standalone question.\n",
    "\n",
    "Chat History:\n",
    "{chat_history}\n",
    "Follow Up Input: {question}\n",
    "Standalone question:\"\"\"\n",
    "\n",
    "condense_question_prompt = ChatPromptTemplate.from_template(condense_question_template)\n",
    "\n",
    "qa_template = \"\"\"\n",
    "You are an assistant for question-answering tasks.\n",
    "Use the following pieces of retrieved context to answer\n",
    "the question. If you don't know the answer, say that you\n",
    "don't know. Use three sentences maximum and keep the\n",
    "answer concise.\n",
    "\n",
    "Chat History:\n",
    "{chat_history}\n",
    "\n",
    "Other context:\n",
    "{context}\n",
    "\n",
    "Question: {question}\n",
    "\"\"\"\n",
    "\n",
    "qa_prompt = ChatPromptTemplate.from_template(qa_template)\n",
    "\n",
    "convo_qa_chain = ConversationalRetrievalChain.from_llm(\n",
    "    llm,\n",
    "    vectorstore.as_retriever(),\n",
    "    condense_question_prompt=condense_question_prompt,\n",
    "    combine_docs_chain_kwargs={\n",
    "        \"prompt\": qa_prompt,\n",
    "    },\n",
    ")\n",
    "\n",
    "convo_qa_chain(\n",
    "    {\n",
    "        \"question\": \"What are autonomous agents?\",\n",
    "        \"chat_history\": \"\",\n",
    "    }\n",
    ")"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "43a8a23c",
   "metadata": {},
   "source": [
    "</details>\n",
    "\n",
    "## LCEL\n",
    "\n",
    "<details open>"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 7,
   "id": "35657a13-ad67-4af1-b1f9-f58606ae43b4",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "{'input': 'What are autonomous agents?',\n",
       " 'chat_history': [],\n",
       " 'context': [Document(metadata={'source': 'https://lilianweng.github.io/posts/2023-06-23-agent/', 'title': \"LLM Powered Autonomous Agents | Lil'Log\", 'description': 'Building agents with LLM (large language model) as its core controller is a cool concept. Several proof-of-concepts demos, such as AutoGPT, GPT-Engineer and BabyAGI, serve as inspiring examples. The potentiality of LLM extends beyond generating well-written copies, stories, essays and programs; it can be framed as a powerful general problem solver.\\nAgent System Overview In a LLM-powered autonomous agent system, LLM functions as the agent’s brain, complemented by several key components:', 'language': 'en'}, page_content='Boiko et al. (2023) also looked into LLM-empowered agents for scientific discovery, to handle autonomous design, planning, and performance of complex scientific experiments. This agent can use tools to browse the Internet, read documentation, execute code, call robotics experimentation APIs and leverage other LLMs.\\nFor example, when requested to \"develop a novel anticancer drug\", the model came up with the following reasoning steps:'),\n",


LangChain has evolved since its initial release, and many of the original "Chain" classes

have been deprecated in favor of the more flexible and powerful frameworks of LCEL and LangGraph.

This guide will help you migrate your existing v0.0 chains to the new abstractions.

:::info How deprecated implementations work

Even though many of these implementations are deprecated, they are **still supported** in the codebase.

However, they are not recommended for new development, and we recommend re-implementing them using the following guides!

To see the planned removal version for each deprecated implementation, check their API reference.

These guides assume some familiarity with the following concepts:

- [LangChain Expression Language](/docs/concepts#langchain-expression-language-lcel)

LangChain maintains a number of legacy abstractions. Many of these can be reimplemented via short combinations of LCEL and LangGraph primitives.

[LCEL](/docs/concepts/#langchain-expression-language-lcel) is designed to streamline the process of building useful apps with LLMs and combining related components. It does this by providing:

1. **A unified interface**: Every LCEL object implements the `Runnable` interface, which defines a common set of invocation methods (`invoke`, `batch`, `stream`, `ainvoke`, ...). This makes it possible to also automatically and consistently support useful operations like streaming of intermediate steps and batching, since every chain composed of LCEL objects is itself an LCEL object.

2. **Composition primitives**: LCEL provides a number of primitives that make it easy to compose chains, parallelize components, add fallbacks, dynamically configure chain internals, and more.

[LangGraph](https://langchain-ai.github.io/langgraph/), built on top of LCEL, allows for performant orchestrations of application components while maintaining concise and readable code. It includes built-in persistence, support for cycles, and prioritizes controllability.

If LCEL grows unwieldy for larger or more complex chains, they may benefit from a LangGraph implementation.

Using these frameworks for existing v0.0 chains confers some advantages:

- The resulting chains typically implement the full `Runnable` interface, including streaming and asynchronous support where appropriate;

- The chains may be more easily extended or modified;

- The parameters of the chain are typically surfaced for easier customization (e.g., prompts) over previous versions, which tended to be subclasses and had opaque parameters and internals.

- If using LangGraph, the chain supports built-in persistence, allowing for conversational experiences via a "memory" of the chat history.

- If using LangGraph, the steps of the chain can be streamed, allowing for greater control and customizability.

The below pages assist with migration from various specific chains to LCEL and LangGraph:

Check out the [LCEL conceptual docs](/docs/concepts/#langchain-expression-language-lcel) and [LangGraph docs](https://langchain-ai.github.io/langgraph/) for more background information.


---
sidebar_position: 1
---

# How to migrate to LangGraph memory

As of the v0.3 release of LangChain, we recommend that LangChain users take advantage of [LangGraph persistence](https://langchain-ai.github.io/langgraph/concepts/persistence/) to incorporate `memory` into their LangChain application.

* Users that rely on `RunnableWithMessageHistory` or `BaseChatMessageHistory` do **not** need to make any changes, but are encouraged to consider using LangGraph for more complex use cases.
* Users that rely on deprecated memory abstractions from LangChain 0.0.x should follow this guide to upgrade to the new LangGraph persistence feature in LangChain 0.3.x.

## Why use LangGraph for memory?

The main advantages of persistence in LangGraph are:

- Built-in support for multiple users and conversations, which is a typical requirement for real-world conversational AI applications.
- Ability to save and resume complex conversations at any point. This helps with:
  - Error recovery
  - Allowing human intervention in AI workflows
  - Exploring different conversation paths ("time travel")
- Full compatibility with both traditional [language models](/docs/concepts/#llms) and modern [chat models](/docs/concepts/#chat-models). Early memory implementations in LangChain weren't designed for newer chat model APIs, causing issues with features like tool-calling. LangGraph memory can persist any custom state.
- Highly customizable, allowing you to fully control how memory works and use different storage backends.

## Evolution of memory in LangChain

The concept of memory has evolved significantly in LangChain since its initial release.

### LangChain 0.0.x memory

Broadly speaking, LangChain 0.0.x memory was used to handle three main use cases:

| Use Case                             | Example                                                                                |
|--------------------------------------|-----------------------------------------------------------------------------------------------------------------------------------|
| Managing conversation history        | Keep only the last `n` turns of the conversation between the user and the AI.                                                     |
| Extraction of structured information | Extract structured information from the conversation history, such as a list of facts learned about the user.                     |
| Composite memory implementations     | Combine multiple memory sources, e.g., a list of known facts about the user along with facts learned during a given conversation. |

While the LangChain 0.0.x memory abstractions were useful, they were limited in their capabilities and not well suited for real-world conversational AI applications. These memory abstractions lacked built-in support for multi-user, multi-conversation scenarios, which are essential for practical conversational AI systems.

Most of these implementations have been officially deprecated in LangChain 0.3.x in favor of LangGraph persistence.

### RunnableWithMessageHistory and BaseChatMessageHistory

:::note
Please see [How to use BaseChatMessageHistory with LangGraph](./chat_history), if you would like to use `BaseChatMessageHistory` (with or without `RunnableWithMessageHistory`) in LangGraph.
:::

As of LangChain v0.1, we started recommending that users rely primarily on [BaseChatMessageHistory](https://python.langchain.com/api_reference/core/runnables/langchain_core.runnables.history.RunnableWithMessageHistory.html#langchain_core.runnables.history.RunnableWithMessageHistory). `BaseChatMessageHistory` serves
as a simple persistence for storing and retrieving messages in a conversation.

At that time, the only option for orchestrating LangChain chains was via [LCEL](https://python.langchain.com/docs/how_to/#langchain-expression-language-lcel). To incorporate memory with `LCEL`, users had to use the [RunnableWithMessageHistory](https://python.langchain.com/api_reference/core/runnables/langchain_core.runnables.history.RunnableWithMessageHistory.html#langchain_core.runnables.history.RunnableWithMessageHistory) interface. While sufficient for basic chat applications, many users found the API unintuitive and challenging to use.

As of LangChain v0.3, we recommend that **new** code takes advantage of LangGraph for both orchestration and persistence:

- Orchestration: In LangGraph, users define [graphs](https://langchain-ai.github.io/langgraph/concepts/low_level/) that specify the flow of the application. This allows users to keep using `LCEL` within individual nodes when `LCEL` is needed, while making it easy to define complex orchestration logic that is more readable and maintainable.
- Persistence: Users can rely on LangGraph's [persistence](https://langchain-ai.github.io/langgraph/concepts/persistence/) to store and retrieve data. LangGraph persistence is extremely flexible and can support a much wider range of use cases than the `RunnableWithMessageHistory` interface.

:::important
If you have been using `RunnableWithMessageHistory` or `BaseChatMessageHistory`, you do not need to make any changes. We do not plan on deprecating either functionality in the near future. This functionality is sufficient for simple chat applications and any code that uses `RunnableWithMessageHistory` will continue to work as expected.
:::

## Migrations

:::info Prerequisites

These guides assume some familiarity with the following concepts:
- [LangGraph](https://langchain-ai.github.io/langgraph/)
- [v0.0.x Memory](https://python.langchain.com/v0.1/docs/modules/memory/)
- [How to add persistence ("memory") to your graph](https://langchain-ai.github.io/langgraph/how-tos/persistence/)
:::

### 1. Managing conversation history

The goal of managing conversation history is to store and retrieve the history in a way that is optimal for a chat model to use.

Often this involves trimming and / or summarizing the conversation history to keep the most relevant parts of the conversation while having the conversation fit inside the context window of the chat model.

Memory classes that fall into this category include:

| Memory Type                       | How to Migrate                                                                                | Description                                                                                |
|-----------------------------------|:------------------------------------------------------------------------------------------------------------------------------------------------------------|---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------|
| `ConversationBufferMemory`        | [Link to Migration Guide](conversation_buffer_memory)                                                                                | A basic memory implementation that simply stores the conversation history.                                                                                |
| `ConversationStringBufferMemory`  | [Link to Migration Guide](conversation_buffer_memory)                                                                                | A special case of `ConversationBufferMemory` designed for LLMs and no longer relevant.                                                                                |
| `ConversationBufferWindowMemory`  | [Link to Migration Guide](conversation_buffer_window_memory)                                                                                | Keeps the last `n` turns of the conversation. Drops the oldest turn when the buffer is full.                                                                                |
| `ConversationTokenBufferMemory`   | [Link to Migration Guide](conversation_buffer_window_memory)                                                                                | Keeps only the most recent messages in the conversation under the constraint that the total number of tokens in the conversation does not exceed a certain limit.                                                   |
| `ConversationSummaryMemory`       | [Link to Migration Guide](conversation_summary_memory)                                                                                | Continually summarizes the conversation history. The summary is updated after each conversation turn. The abstraction returns the summary of the conversation history.                                              |
| `ConversationSummaryBufferMemory` | [Link to Migration Guide](conversation_summary_memory)                                                                                | Provides a running summary of the conversation together with the most recent messages in the conversation under the constraint that the total number of tokens in the conversation does not exceed a certain limit. |
| `VectorStoreRetrieverMemory`      | See related [long-term memory agent tutorial](long_term_memory_agent) | Stores the conversation history in a vector store and retrieves the most relevant parts of past conversation based on the input.                                                                                |


{
 "cells": [
  {
   "cell_type": "markdown",
   "id": "ce8457ed-c0b1-4a74-abbd-9d3d2211270f",
   "metadata": {},
   "source": [
    "# Migrating off ConversationBufferWindowMemory or ConversationTokenBufferMemory\n",
    "\n",
    "Follow this guide if you're trying to migrate off one of the old memory classes listed below:\n",
    "\n",
    "\n",
    "| Memory Type                      | Description                                                                                |\n",
    "|----------------------------------|-------------------------------------------------------------------------------------------------------------------------------------------------------------------|\n",
    "| `ConversationBufferWindowMemory` | Keeps the last `n` messages of the conversation. Drops the oldest messages when there are more than `n` messages.                                                                      |\n",
    "| `ConversationTokenBufferMemory`  | Keeps only the most recent messages in the conversation under the constraint that the total number of tokens in the conversation does not exceed a certain limit. |\n",
    "\n",
    "`ConversationBufferWindowMemory` and `ConversationTokenBufferMemory` apply additional processing on top of the raw conversation history to trim the conversation history to a size that fits inside the context window of a chat model. \n",
    "\n",
    "This processing functionality can be accomplished using LangChain's built-in [trim_messages](https://python.langchain.com/api_reference/core/messages/langchain_core.messages.utils.trim_messages.html) function."
   ]
  },
  {
   "cell_type": "markdown",
   "id": "79935247-acc7-4a05-a387-5d72c9c8c8cb",
   "metadata": {},
   "source": [
    ":::important\n",
    "\n",
    "We’ll begin by exploring a straightforward method that involves applying processing logic to the entire conversation history.\n",
    "\n",
    "While this approach is easy to implement, it has a downside: as the conversation grows, so does the latency, since the logic is re-applied to all previous exchanges in the conversation at each turn.\n",
    "\n",
    "More advanced strategies focus on incrementally updating the conversation history to avoid redundant processing.\n",
    "\n",
    "For instance, the langgraph [how-to guide on summarization](https://langchain-ai.github.io/langgraph/how-tos/memory/add-summary-conversation-history/) demonstrates\n",
    "how to maintain a running summary of the conversation while discarding older messages, ensuring they aren't re-processed during later turns.\n",
    ":::"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "d07f9459-9fb6-4942-99c9-64558aedd7d4",
   "metadata": {},
   "source": [
    "## Set up"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 1,
   "id": "b99b47ec",
   "metadata": {},
   "outputs": [],
   "source": [
    "%%capture --no-stderr\n",
    "%pip install --upgrade --quiet langchain-openai langchain"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 1,
   "id": "7127478f-4413-48be-bfec-d0cd91b8cf70",
   "metadata": {},
   "outputs": [],
   "source": [
    "import os\n",
    "from getpass import getpass\n",
    "\n",
    "if \"OPENAI_API_KEY\" not in os.environ:\n",
    "    os.environ[\"OPENAI_API_KEY\"] = getpass()"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "d6a7bc93-21a9-44c8-842e-9cc82f1ada7c",
   "metadata": {},
   "source": [
    "## Legacy usage with LLMChain / Conversation Chain\n",
    "\n",
    "<details open>"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 3,
   "id": "371616e1-ca41-4a57-99e0-5fbf7d63f2ad",
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "{'text': 'Nice to meet you, Bob! How can I assist you today?', 'chat_history': []}\n",
      "{'text': 'Your name is Bob. How can I assist you further, Bob?', 'chat_history': [HumanMessage(content='my name is bob', additional_kwargs={}, response_metadata={}), AIMessage(content='Nice to meet you, Bob! How can I assist you today?', additional_kwargs={}, response_metadata={})]}\n"
     ]
    }
   ],
   "source": [
    "from langchain.chains import LLMChain\n",
    "from langchain.memory import ConversationBufferWindowMemory\n",
    "from langchain_core.messages import SystemMessage\n",
    "from langchain_core.prompts import ChatPromptTemplate\n",
    "from langchain_core.prompts.chat import (\n",
    "    ChatPromptTemplate,\n",
    "    HumanMessagePromptTemplate,\n",
    "    MessagesPlaceholder,\n",
    ")\n",
    "from langchain_openai import ChatOpenAI\n",
    "\n",
    "prompt = ChatPromptTemplate(\n",
    "    [\n",
    "        SystemMessage(content=\"You are a helpful assistant.\"),\n",
    "        MessagesPlaceholder(variable_name=\"chat_history\"),\n",
    "        HumanMessagePromptTemplate.from_template(\"{text}\"),\n",
    "    ]\n",
    ")\n",
    "\n",
    "# highlight-start\n",
    "memory = ConversationBufferWindowMemory(memory_key=\"chat_history\", return_messages=True)\n",
    "# highlight-end\n",
    "\n",
    "legacy_chain = LLMChain(\n",
    "    llm=ChatOpenAI(),\n",
    "    prompt=prompt,\n",
    "    # highlight-next-line\n",
    "    memory=memory,\n",
    ")\n",
    "\n",
    "legacy_result = legacy_chain.invoke({\"text\": \"my name is bob\"})\n",
    "print(legacy_result)\n",
    "\n",
    "legacy_result = legacy_chain.invoke({\"text\": \"what was my name\"})\n",
    "print(legacy_result)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "f48cac47-c8b6-444c-8e1b-f7115c0b2d8d",
   "metadata": {},
   "source": [
    "</details>\n",
    "\n",
    "## Reimplementing ConversationBufferWindowMemory logic\n",
    "\n",
    "Let's first create appropriate logic to process the conversation history, and then we'll see how to integrate it into an application. You can later replace this basic setup with more advanced logic tailored to your specific needs.\n",
    "\n",
    "We'll use `trim_messages` to implement logic that keeps the last `n` messages of the conversation. It will drop the oldest messages when the number of messages exceeds `n`.\n",
    "\n",
    "In addition, we will also keep the system message if it's present -- when present, it's the first message in a conversation that includes instructions for the chat model."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 4,
   "id": "0a92b3f3-0315-46ac-bb28-d07398dd23ea",
   "metadata": {},
   "outputs": [],
   "source": [
    "from langchain_core.messages import (\n",
    "    AIMessage,\n",
    "    BaseMessage,\n",
    "    HumanMessage,\n",
    "    SystemMessage,\n",
    "    trim_messages,\n",
    ")\n",
    "from langchain_openai import ChatOpenAI\n",
    "\n",
    "messages = [\n",
    "    SystemMessage(\"you're a good assistant, you always respond with a joke.\"),\n",
    "    HumanMessage(\"i wonder why it's called langchain\"),\n",
    "    AIMessage(\n",
    "        'Well, I guess they thought \"WordRope\" and \"SentenceString\" just didn\\'t have the same ring to it!'\n",
    "    ),\n",
    "    HumanMessage(\"and who is harrison chasing anyways\"),\n",
    "    AIMessage(\n",
    "        \"Hmmm let me think.\\n\\nWhy, he's probably chasing after the last cup of coffee in the office!\"\n",
    "    ),\n",
    "    HumanMessage(\"why is 42 always the answer?\"),\n",
    "    AIMessage(\n",
    "        \"Because it’s the only number that’s constantly right, even when it doesn’t add up!\"\n",
    "    ),\n",
    "    HumanMessage(\"What did the cow say?\"),\n",
    "]"
   ]
  },
  {
   "cell_type": "code",


"        include_system=True,\n",
    "        allow_partial=False,\n",
    "    )\n",
    "\n",
    "    # highlight-end\n",
    "    response = model.invoke(selected_messages)\n",
    "    # We return a list, because this will get added to the existing list\n",
    "    return {\"messages\": response}\n",
    "\n",
    "\n",
    "# Define the two nodes we will cycle between\n",
    "workflow.add_edge(START, \"model\")\n",
    "workflow.add_node(\"model\", call_model)\n",
    "\n",
    "\n",
    "# Adding memory is straight forward in langgraph!\n",
    "# highlight-next-line\n",
    "memory = MemorySaver()\n",
    "\n",
    "app = workflow.compile(\n",
    "    # highlight-next-line\n",
    "    checkpointer=memory\n",
    ")\n",
    "\n",
    "\n",
    "# The thread id is a unique key that identifies\n",
    "# this particular conversation.\n",
    "# We'll just generate a random uuid here.\n",
    "thread_id = uuid.uuid4()\n",
    "# highlight-next-line\n",
    "config = {\"configurable\": {\"thread_id\": thread_id}}\n",
    "\n",
    "input_message = HumanMessage(content=\"hi! I'm bob\")\n",
    "for event in app.stream({\"messages\": [input_message]}, config, stream_mode=\"values\"):\n",
    "    event[\"messages\"][-1].pretty_print()\n",
    "\n",
    "# Here, let's confirm that the AI remembers our name!\n",
    "config = {\"configurable\": {\"thread_id\": thread_id}}\n",
    "input_message = HumanMessage(content=\"what was my name?\")\n",
    "for event in app.stream({\"messages\": [input_message]}, config, stream_mode=\"values\"):\n",
    "    event[\"messages\"][-1].pretty_print()"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "84229e2e-a578-4b21-840a-814223406402",
   "metadata": {},
   "source": [
    "</details>\n",
    "\n",
    "## Usage with a pre-built langgraph agent\n",
    "\n",
    "This example shows usage of an Agent Executor with a pre-built agent constructed using the [create_tool_calling_agent](https://api.python.langchain.com/en/latest/agents/langchain.agents.tool_calling_agent.base.create_tool_calling_agent.html) function.\n",
    "\n",
    "If you are using one of the [old LangChain pre-built agents](https://python.langchain.com/v0.1/docs/modules/agents/agent_types/), you should be able\n",
    "to replace that code with the new [langgraph pre-built agent](https://langchain-ai.github.io/langgraph/how-tos/create-react-agent/) which leverages\n",
    "native tool calling capabilities of chat models and will likely work better out of the box.\n",
    "\n",
    "<details open>"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 8,
   "id": "f671db87-8f01-453e-81fd-4e603140a512",
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "================================\u001b[1m Human Message \u001b[0m=================================\n",
      "\n",
      "hi! I'm bob. What is my age?\n",
      "==================================\u001b[1m Ai Message \u001b[0m==================================\n",
      "Tool Calls:\n",
      "  get_user_age (call_jsMvoIFv970DhqqLCJDzPKsp)\n",
      " Call ID: call_jsMvoIFv970DhqqLCJDzPKsp\n",
      "  Args:\n",
      "    name: bob\n",
      "=================================\u001b[1m Tool Message \u001b[0m=================================\n",
      "Name: get_user_age\n",
      "\n",
      "42 years old\n",
      "==================================\u001b[1m Ai Message \u001b[0m==================================\n",
      "\n",
      "Bob, you are 42 years old.\n",
      "================================\u001b[1m Human Message \u001b[0m=================================\n",
      "\n",
      "do you remember my name?\n",
      "==================================\u001b[1m Ai Message \u001b[0m==================================\n",
      "\n",
      "Yes, your name is Bob.\n"
     ]
    }
   ],
   "source": [
    "import uuid\n",
    "\n",
    "from langchain_core.messages import (\n",
    "    AIMessage,\n",
    "    BaseMessage,\n",
    "    HumanMessage,\n",
    "    SystemMessage,\n",
    "    trim_messages,\n",
    ")\n",
    "from langchain_core.tools import tool\n",
    "from langchain_openai import ChatOpenAI\n",
    "from langgraph.checkpoint.memory import MemorySaver\n",
    "from langgraph.prebuilt import create_react_agent\n",
    "\n",
    "\n",
    "@tool\n",
    "def get_user_age(name: str) -> str:\n",
    "    \"\"\"Use this tool to find the user's age.\"\"\"\n",
    "    # This is a placeholder for the actual implementation\n",
    "    if \"bob\" in name.lower():\n",
    "        return \"42 years old\"\n",
    "    return \"41 years old\"\n",
    "\n",
    "\n",
    "memory = MemorySaver()\n",
    "model = ChatOpenAI()\n",
    "\n",
    "\n",
    "# highlight-start\n",
    "def state_modifier(state) -> list[BaseMessage]:\n",
    "    \"\"\"Given the agent state, return a list of messages for the chat model.\"\"\"\n",
    "    # We're using the message processor defined above.\n",
    "    return trim_messages(\n",
    "        state[\"messages\"],\n",
    "        token_counter=len,  # <-- len will simply count the number of messages rather than tokens\n",
    "        max_tokens=5,  # <-- allow up to 5 messages.\n",
    "        strategy=\"last\",\n",
    "        # Most chat models expect that chat history starts with either:\n",
    "        # (1) a HumanMessage or\n",
    "        # (2) a SystemMessage followed by a HumanMessage\n",
    "        # start_on=\"human\" makes sure we produce a valid chat history\n",
    "        start_on=\"human\",\n",
    "        # Usually, we want to keep the SystemMessage\n",
    "        # if it's present in the original history.\n",
    "        # The SystemMessage has special instructions for the model.\n",
    "        include_system=True,\n",
    "        allow_partial=False,\n",
    "    )\n",
    "\n",
    "\n",
    "# highlight-end\n",
    "\n",
    "app = create_react_agent(\n",
    "    model,\n",
    "    tools=[get_user_age],\n",
    "    checkpointer=memory,\n",
    "    # highlight-next-line\n",
    "    state_modifier=state_modifier,\n",
    ")\n",
    "\n",
    "# The thread id is a unique key that identifies\n",
    "# this particular conversation.\n",
    "# We'll just generate a random uuid here.\n",
    "thread_id = uuid.uuid4()\n",
    "config = {\"configurable\": {\"thread_id\": thread_id}}\n",
    "\n",
    "# Tell the AI that our name is Bob, and ask it to use a tool to confirm\n",
    "# that it's capable of working like an agent.\n",
    "input_message = HumanMessage(content=\"hi! I'm bob. What is my age?\")\n",
    "\n",
    "for event in app.stream({\"messages\": [input_message]}, config, stream_mode=\"values\"):\n",
    "    event[\"messages\"][-1].pretty_print()\n",
    "\n",
    "# Confirm that the chat bot has access to previous conversation\n",
    "# and can respond to the user saying that the user's name is Bob.\n",
    "input_message = HumanMessage(content=\"do you remember my name?\")\n",
    "\n",
    "for event in app.stream({\"messages\": [input_message]}, config, stream_mode=\"values\"):\n",
    "    event[\"messages\"][-1].pretty_print()"
   ]
  },
  {
   "attachments": {},
   "cell_type": "markdown",
   "id": "f4d16e09-1d90-4153-8576-6d3996cb5a6c",


# INVALID_PROMPT_INPUT

A [prompt template](/docs/concepts#prompt-templates) received missing or invalid input variables.

## Troubleshooting

The following may help resolve this error:

- Double-check your prompt template to ensure that it is correct.
  - If you are using the default f-string format and you are using curly braces `{` anywhere in your template, they should be double escaped like this: `{{` (and if you want to render a double curly brace, you should use four curly braces: `{{{{`).
- If you are using a [`MessagesPlaceholder`](/docs/concepts/#messagesplaceholder), make sure that you are passing in an array of messages or message-like objects.
  - If you are using shorthand tuples to declare your prompt template, make sure that the variable name is wrapped in curly braces (`["placeholder", "{messages}"]`).
- Try viewing the inputs into your prompt template using [LangSmith](https://docs.smith.langchain.com/) or log statements to confirm they appear as expected.
- If you are pulling a prompt from the [LangChain Prompt Hub](https://smith.langchain.com/prompts), try pulling and logging it or running it in isolation with a sample input to confirm that it is what you expect.


{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# MESSAGE_COERCION_FAILURE\n",
    "\n",
    "Instead of always requiring instances of `BaseMessage`, several modules in LangChain take `MessageLikeRepresentation`, which is defined as:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 4,
   "metadata": {},
   "outputs": [],
   "source": [
    "from typing import Union\n",
    "\n",
    "from langchain_core.prompts.chat import (\n",
    "    BaseChatPromptTemplate,\n",
    "    BaseMessage,\n",
    "    BaseMessagePromptTemplate,\n",
    ")\n",
    "\n",
    "MessageLikeRepresentation = Union[\n",
    "    Union[BaseMessagePromptTemplate, BaseMessage, BaseChatPromptTemplate],\n",
    "    tuple[\n",
    "        Union[str, type],\n",
    "        Union[str, list[dict], list[object]],\n",
    "    ],\n",
    "    str,\n",
    "]"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "These include OpenAI style message objects (`{ role: \"user\", content: \"Hello world!\" }`),\n",
    "tuples, and plain strings (which are converted to [`HumanMessages`](/docs/concepts#humanmessage)).\n",
    "\n",
    "If one of these modules receives a value outside of one of these formats, you will receive an error like the following:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 5,
   "metadata": {},
   "outputs": [
    {
     "ename": "ValueError",
     "evalue": "Message dict must contain 'role' and 'content' keys, got {'role': 'HumanMessage', 'random_field': 'random value'}",
     "output_type": "error",
     "traceback": [
      "\u001b[0;31m---------------------------------------------------------------------------\u001b[0m",
      "\u001b[0;31mKeyError\u001b[0m                                  Traceback (most recent call last)",
      "File \u001b[0;32m~/langchain/oss-py/libs/core/langchain_core/messages/utils.py:318\u001b[0m, in \u001b[0;36m_convert_to_message\u001b[0;34m(message)\u001b[0m\n\u001b[1;32m    317\u001b[0m     \u001b[38;5;66;03m# None msg content is not allowed\u001b[39;00m\n\u001b[0;32m--> 318\u001b[0m     msg_content \u001b[38;5;241m=\u001b[39m \u001b[43mmsg_kwargs\u001b[49m\u001b[38;5;241;43m.\u001b[39;49m\u001b[43mpop\u001b[49m\u001b[43m(\u001b[49m\u001b[38;5;124;43m\"\u001b[39;49m\u001b[38;5;124;43mcontent\u001b[39;49m\u001b[38;5;124;43m\"\u001b[39;49m\u001b[43m)\u001b[49m \u001b[38;5;129;01mor\u001b[39;00m \u001b[38;5;124m\"\u001b[39m\u001b[38;5;124m\"\u001b[39m\n\u001b[1;32m    319\u001b[0m \u001b[38;5;28;01mexcept\u001b[39;00m \u001b[38;5;167;01mKeyError\u001b[39;00m \u001b[38;5;28;01mas\u001b[39;00m e:\n",
      "\u001b[0;31mKeyError\u001b[0m: 'content'",
      "\nThe above exception was the direct cause of the following exception:\n",
      "\u001b[0;31mValueError\u001b[0m                                Traceback (most recent call last)",
      "Cell \u001b[0;32mIn[5], line 10\u001b[0m\n\u001b[1;32m      3\u001b[0m uncoercible_message \u001b[38;5;241m=\u001b[39m {\n\u001b[1;32m      4\u001b[0m     \u001b[38;5;124m\"\u001b[39m\u001b[38;5;124mrole\u001b[39m\u001b[38;5;124m\"\u001b[39m: \u001b[38;5;124m\"\u001b[39m\u001b[38;5;124mHumanMessage\u001b[39m\u001b[38;5;124m\"\u001b[39m,\n\u001b[1;32m      5\u001b[0m     \u001b[38;5;124m\"\u001b[39m\u001b[38;5;124mrandom_field\u001b[39m\u001b[38;5;124m\"\u001b[39m: \u001b[38;5;124m\"\u001b[39m\u001b[38;5;124mrandom value\u001b[39m\u001b[38;5;124m\"\u001b[39m\n\u001b[1;32m      6\u001b[0m }\n\u001b[1;32m      8\u001b[0m model \u001b[38;5;241m=\u001b[39m ChatAnthropic(model\u001b[38;5;241m=\u001b[39m\u001b[38;5;124m\"\u001b[39m\u001b[38;5;124mclaude-3-5-sonnet-20240620\u001b[39m\u001b[38;5;124m\"\u001b[39m)\n\u001b[0;32m---> 10\u001b[0m \u001b[43mmodel\u001b[49m\u001b[38;5;241;43m.\u001b[39;49m\u001b[43minvoke\u001b[49m\u001b[43m(\u001b[49m\u001b[43m[\u001b[49m\u001b[43muncoercible_message\u001b[49m\u001b[43m]\u001b[49m\u001b[43m)\u001b[49m\n",


"File \u001b[0;32m~/langchain/oss-py/libs/core/langchain_core/language_models/chat_models.py:267\u001b[0m, in \u001b[0;36mBaseChatModel._convert_input\u001b[0;34m(self, input)\u001b[0m\n\u001b[1;32m    265\u001b[0m     \u001b[38;5;28;01mreturn\u001b[39;00m StringPromptValue(text\u001b[38;5;241m=\u001b[39m\u001b[38;5;28minput\u001b[39m)\n\u001b[1;32m    266\u001b[0m \u001b[38;5;28;01melif\u001b[39;00m \u001b[38;5;28misinstance\u001b[39m(\u001b[38;5;28minput\u001b[39m, Sequence):\n\u001b[0;32m--> 267\u001b[0m     \u001b[38;5;28;01mreturn\u001b[39;00m ChatPromptValue(messages\u001b[38;5;241m=\u001b[39m\u001b[43mconvert_to_messages\u001b[49m\u001b[43m(\u001b[49m\u001b[38;5;28;43minput\u001b[39;49m\u001b[43m)\u001b[49m)\n\u001b[1;32m    268\u001b[0m \u001b[38;5;28;01melse\u001b[39;00m:\n\u001b[1;32m    269\u001b[0m     msg \u001b[38;5;241m=\u001b[39m (\n\u001b[1;32m    270\u001b[0m         \u001b[38;5;124mf\u001b[39m\u001b[38;5;124m\"\u001b[39m\u001b[38;5;124mInvalid input type \u001b[39m\u001b[38;5;132;01m{\u001b[39;00m\u001b[38;5;28mtype\u001b[39m(\u001b[38;5;28minput\u001b[39m)\u001b[38;5;132;01m}\u001b[39;00m\u001b[38;5;124m. \u001b[39m\u001b[38;5;124m\"\u001b[39m\n\u001b[1;32m    271\u001b[0m         \u001b[38;5;124m\"\u001b[39m\u001b[38;5;124mMust be a PromptValue, str, or list of BaseMessages.\u001b[39m\u001b[38;5;124m\"\u001b[39m\n\u001b[1;32m    272\u001b[0m     )\n",
      "File \u001b[0;32m~/langchain/oss-py/libs/core/langchain_core/messages/utils.py:348\u001b[0m, in \u001b[0;36mconvert_to_messages\u001b[0;34m(messages)\u001b[0m\n\u001b[1;32m    346\u001b[0m \u001b[38;5;28;01mif\u001b[39;00m \u001b[38;5;28misinstance\u001b[39m(messages, PromptValue):\n\u001b[1;32m    347\u001b[0m     \u001b[38;5;28;01mreturn\u001b[39;00m messages\u001b[38;5;241m.\u001b[39mto_messages()\n\u001b[0;32m--> 348\u001b[0m \u001b[38;5;28;01mreturn\u001b[39;00m \u001b[43m[\u001b[49m\u001b[43m_convert_to_message\u001b[49m\u001b[43m(\u001b[49m\u001b[43mm\u001b[49m\u001b[43m)\u001b[49m\u001b[43m \u001b[49m\u001b[38;5;28;43;01mfor\u001b[39;49;00m\u001b[43m \u001b[49m\u001b[43mm\u001b[49m\u001b[43m \u001b[49m\u001b[38;5;129;43;01min\u001b[39;49;00m\u001b[43m \u001b[49m\u001b[43mmessages\u001b[49m\u001b[43m]\u001b[49m\n",
      "File \u001b[0;32m~/langchain/oss-py/libs/core/langchain_core/messages/utils.py:348\u001b[0m, in \u001b[0;36m<listcomp>\u001b[0;34m(.0)\u001b[0m\n\u001b[1;32m    346\u001b[0m \u001b[38;5;28;01mif\u001b[39;00m \u001b[38;5;28misinstance\u001b[39m(messages, PromptValue):\n\u001b[1;32m    347\u001b[0m     \u001b[38;5;28;01mreturn\u001b[39;00m messages\u001b[38;5;241m.\u001b[39mto_messages()\n\u001b[0;32m--> 348\u001b[0m \u001b[38;5;28;01mreturn\u001b[39;00m [\u001b[43m_convert_to_message\u001b[49m\u001b[43m(\u001b[49m\u001b[43mm\u001b[49m\u001b[43m)\u001b[49m \u001b[38;5;28;01mfor\u001b[39;00m m \u001b[38;5;129;01min\u001b[39;00m messages]\n",


"File \u001b[0;32m~/langchain/oss-py/libs/core/langchain_core/messages/utils.py:321\u001b[0m, in \u001b[0;36m_convert_to_message\u001b[0;34m(message)\u001b[0m\n\u001b[1;32m    319\u001b[0m     \u001b[38;5;28;01mexcept\u001b[39;00m \u001b[38;5;167;01mKeyError\u001b[39;00m \u001b[38;5;28;01mas\u001b[39;00m e:\n\u001b[1;32m    320\u001b[0m         msg \u001b[38;5;241m=\u001b[39m \u001b[38;5;124mf\u001b[39m\u001b[38;5;124m\"\u001b[39m\u001b[38;5;124mMessage dict must contain \u001b[39m\u001b[38;5;124m'\u001b[39m\u001b[38;5;124mrole\u001b[39m\u001b[38;5;124m'\u001b[39m\u001b[38;5;124m and \u001b[39m\u001b[38;5;124m'\u001b[39m\u001b[38;5;124mcontent\u001b[39m\u001b[38;5;124m'\u001b[39m\u001b[38;5;124m keys, got \u001b[39m\u001b[38;5;132;01m{\u001b[39;00mmessage\u001b[38;5;132;01m}\u001b[39;00m\u001b[38;5;124m\"\u001b[39m\n\u001b[0;32m--> 321\u001b[0m         \u001b[38;5;28;01mraise\u001b[39;00m \u001b[38;5;167;01mValueError\u001b[39;00m(msg) \u001b[38;5;28;01mfrom\u001b[39;00m \u001b[38;5;21;01me\u001b[39;00m\n\u001b[1;32m    322\u001b[0m     _message \u001b[38;5;241m=\u001b[39m _create_message_from_message_type(\n\u001b[1;32m    323\u001b[0m         msg_type, msg_content, \u001b[38;5;241m*\u001b[39m\u001b[38;5;241m*\u001b[39mmsg_kwargs\n\u001b[1;32m    324\u001b[0m     )\n\u001b[1;32m    325\u001b[0m \u001b[38;5;28;01melse\u001b[39;00m:\n",
      "\u001b[0;31mValueError\u001b[0m: Message dict must contain 'role' and 'content' keys, got {'role': 'HumanMessage', 'random_field': 'random value'}"
     ]
    }
   ],
   "source": [
    "from langchain_anthropic import ChatAnthropic\n",
    "\n",
    "uncoercible_message = {\"role\": \"HumanMessage\", \"random_field\": \"random value\"}\n",
    "\n",
    "model = ChatAnthropic(model=\"claude-3-5-sonnet-20240620\")\n",
    "\n",
    "model.invoke([uncoercible_message])"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Troubleshooting\n",
    "\n",
    "The following may help resolve this error:\n",
    "\n",
    "- Ensure that all inputs to chat models are an array of LangChain message classes or a supported message-like.\n",
    "  - Check that there is no stringification or other unexpected transformation occuring.\n",
    "- Check the error's stack trace and add log or debugger statements."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": []
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": ".venv",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.11.4"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 2
}


# MODEL_AUTHENTICATION

Your model provider is denying you access to their service.

## Troubleshooting

The following may help resolve this error:

- Confirm that your API key or other credentials are correct.
- If you are relying on an environment variable to authenticate, confirm that the variable name is correct and that it has a value set.
  - Note that environment variables can also be set by packages like `dotenv`.
  - For models, you can try explicitly passing an `api_key` parameter to rule out any environment variable issues like this:

```python
model = ChatOpenAI(api_key="YOUR_KEY_HERE")
```

- If you are using a proxy or other custom endpoint, make sure that your custom provider does not expect an alternative authentication scheme.


This guide assumes familiarity with the following concepts:

- [LangChain Expression Language (LCEL)](/docs/concepts/#langchain-expression-language)

Once you create a runnable with [LangChain Expression Language](/docs/concepts/#langchain-expression-language), you may often want to inspect it to get a better sense for what is going on. This notebook covers some methods for doing so.

This guide shows some ways you can programmatically introspect the internal steps of chains. If you are instead interested in debugging issues in your chain, see [this section](/docs/how_to/debugging) instead.

First, let's create an example chain. We will create one that does retrieval:

%pip install -qU langchain langchain-openai faiss-cpu tiktoken

from langchain_community.vectorstores import FAISS

from langchain_core.output_parsers import StrOutputParser

from langchain_core.prompts import ChatPromptTemplate

from langchain_core.runnables import RunnablePassthrough

from langchain_openai import ChatOpenAI, OpenAIEmbeddings

["harrison worked at kensho"], embedding=OpenAIEmbeddings()

template = """Answer the question based only on the following context:

{"context": retriever, "question": RunnablePassthrough()}

You can use the `get_graph()` method to get a graph representation of the runnable:

While that is not super legible, you can use the `print_ascii()` method to show that graph in a way that's easier to understand:

stdout

| Parallel<context,question>Input |

| VectorStoreRetriever |              | Passthrough |

| Parallel<context,question>Output |

| ChatPromptTemplate |

| ChatOpenAI |

| StrOutputParser |

| StrOutputParserOutput |

You may want to see just the prompts that are used in a chain with the `get_prompts()` method:

[ChatPromptTemplate(input_variables=['context', 'question'], messages=[HumanMessagePromptTemplate(prompt=PromptTemplate(input_variables=['context', 'question'], template='Answer the question based only on the following context:\n{context}\n\nQuestion: {question}\n'))])]

You've now learned how to introspect your composed LCEL chains.

Next, check out the other how-to guides on runnables in this section, or the related how-to guide on [debugging your chains](/docs/how_to/debugging).


{
 "cells": [
  {
   "cell_type": "raw",
   "metadata": {},
   "source": [
    "---\n",
    "sidebar_position: 2\n",
    "---"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# How to add retrieval to chatbots\n",
    "\n",
    "Retrieval is a common technique chatbots use to augment their responses with data outside a chat model's training data. This section will cover how to implement retrieval in the context of chatbots, but it's worth noting that retrieval is a very subtle and deep topic - we encourage you to explore [other parts of the documentation](/docs/how_to#qa-with-rag) that go into greater depth!\n",
    "\n",
    "## Setup\n",
    "\n",
    "You'll need to install a few packages, and have your OpenAI API key set as an environment variable named `OPENAI_API_KEY`:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 1,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "\u001b[33mWARNING: You are using pip version 22.0.4; however, version 23.3.2 is available.\n",
      "You should consider upgrading via the '/Users/jacoblee/.pyenv/versions/3.10.5/bin/python -m pip install --upgrade pip' command.\u001b[0m\u001b[33m\n",
      "\u001b[0mNote: you may need to restart the kernel to use updated packages.\n"
     ]
    },
    {
     "data": {
      "text/plain": [
       "True"
      ]
     },
     "execution_count": 1,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "%pip install -qU langchain langchain-openai langchain-chroma beautifulsoup4\n",
    "\n",
    "# Set env var OPENAI_API_KEY or load from a .env file:\n",
    "import dotenv\n",
    "\n",
    "dotenv.load_dotenv()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Let's also set up a chat model that we'll use for the below examples."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 2,
   "metadata": {},
   "outputs": [],
   "source": [
    "from langchain_openai import ChatOpenAI\n",
    "\n",
    "chat = ChatOpenAI(model=\"gpt-4o-mini\", temperature=0.2)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Creating a retriever\n",
    "\n",
    "We'll use [the LangSmith documentation](https://docs.smith.langchain.com/overview) as source material and store the content in a vectorstore for later retrieval. Note that this example will gloss over some of the specifics around parsing and storing a data source - you can see more [in-depth documentation on creating retrieval systems here](/docs/how_to#qa-with-rag).\n",
    "\n",
    "Let's use a document loader to pull text from the docs:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 3,
   "metadata": {},
   "outputs": [],
   "source": [
    "from langchain_community.document_loaders import WebBaseLoader\n",
    "\n",
    "loader = WebBaseLoader(\"https://docs.smith.langchain.com/overview\")\n",
    "data = loader.load()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Next, we split it into smaller chunks that the LLM's context window can handle and store it in a vector database:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 4,
   "metadata": {},
   "outputs": [],
   "source": [
    "from langchain_text_splitters import RecursiveCharacterTextSplitter\n",
    "\n",
    "text_splitter = RecursiveCharacterTextSplitter(chunk_size=500, chunk_overlap=0)\n",
    "all_splits = text_splitter.split_documents(data)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Then we embed and store those chunks in a vector database:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 5,
   "metadata": {},
   "outputs": [],
   "source": [
    "from langchain_chroma import Chroma\n",
    "from langchain_openai import OpenAIEmbeddings\n",
    "\n",
    "vectorstore = Chroma.from_documents(documents=all_splits, embedding=OpenAIEmbeddings())"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "And finally, let's create a retriever from our initialized vectorstore:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 6,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "[Document(page_content='Skip to main content🦜🛠 LangSmith DocsPython DocsJS/TS DocsSearchGo to AppLangSmithOverviewTracingTesting & EvaluationOrganizationsHubLangSmith CookbookOverviewOn this pageLangSmith Overview and User GuideBuilding reliable LLM applications can be challenging. LangChain simplifies the initial setup, but there is still work needed to bring the performance of prompts, chains and agents up the level where they are reliable enough to be used in production.Over the past two months, we at LangChain', metadata={'description': 'Building reliable LLM applications can be challenging. LangChain simplifies the initial setup, but there is still work needed to bring the performance of prompts, chains and agents up the level where they are reliable enough to be used in production.', 'language': 'en', 'source': 'https://docs.smith.langchain.com/overview', 'title': 'LangSmith Overview and User Guide | 🦜🛠 LangSmith'}),\n",
       " Document(page_content='LangSmith Overview and User Guide | 🦜🛠 LangSmith', metadata={'description': 'Building reliable LLM applications can be challenging. LangChain simplifies the initial setup, but there is still work needed to bring the performance of prompts, chains and agents up the level where they are reliable enough to be used in production.', 'language': 'en', 'source': 'https://docs.smith.langchain.com/overview', 'title': 'LangSmith Overview and User Guide | 🦜🛠 LangSmith'}),\n",
       " Document(page_content='You can also quickly edit examples and add them to datasets to expand the surface area of your evaluation sets or to fine-tune a model for improved quality or reduced costs.Monitoring\\u200bAfter all this, your app might finally ready to go in production. LangSmith can also be used to monitor your application in much the same way that you used for debugging. You can log all traces, visualize latency and token usage statistics, and troubleshoot specific issues as they arise. Each run can also be', metadata={'description': 'Building reliable LLM applications can be challenging. LangChain simplifies the initial setup, but there is still work needed to bring the performance of prompts, chains and agents up the level where they are reliable enough to be used in production.', 'language': 'en', 'source': 'https://docs.smith.langchain.com/overview', 'title': 'LangSmith Overview and User Guide | 🦜🛠 LangSmith'}),\n",
       " Document(page_content=\"does that affect the output?\\u200bSo you notice a bad output, and you go into LangSmith to see what's going on. You find the faulty LLM call and are now looking at the exact input. You want to try changing a word or a phrase to see what happens -- what do you do?We constantly ran into this issue. Initially, we copied the prompt to a playground of sorts. But this got annoying, so we built a playground of our own! When examining an LLM call, you can click the Open in Playground button to access this\", metadata={'description': 'Building reliable LLM applications can be challenging. LangChain simplifies the initial setup, but there is still work needed to bring the performance of prompts, chains and agents up the level where they are reliable enough to be used in production.', 'language': 'en', 'source': 'https://docs.smith.langchain.com/overview', 'title': 'LangSmith Overview and User Guide | 🦜🛠 LangSmith'})]"
      ]
     },
     "execution_count": 6,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# k is the number of chunks to retrieve\n",
    "retriever = vectorstore.as_retriever(k=4)\n",
    "\n",


"docs = retriever.invoke(\"Can LangSmith help test my LLM applications?\")\n",
    "\n",
    "docs"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "We can see that invoking the retriever above results in some parts of the LangSmith docs that contain information about testing that our chatbot can use as context when answering questions. And now we've got a retriever that can return related data from the LangSmith docs!\n",
    "\n",
    "## Document chains\n",
    "\n",
    "Now that we have a retriever that can return LangChain docs, let's create a chain that can use them as context to answer questions. We'll use a `create_stuff_documents_chain` helper function to \"stuff\" all of the input documents into the prompt. It will also handle formatting the docs as strings.\n",
    "\n",
    "In addition to a chat model, the function also expects a prompt that has a `context` variables, as well as a placeholder for chat history messages named `messages`. We'll create an appropriate prompt and pass it as shown below:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 7,
   "metadata": {},
   "outputs": [],
   "source": [
    "from langchain.chains.combine_documents import create_stuff_documents_chain\n",
    "from langchain_core.prompts import ChatPromptTemplate, MessagesPlaceholder\n",
    "\n",
    "SYSTEM_TEMPLATE = \"\"\"\n",
    "Answer the user's questions based on the below context. \n",
    "If the context doesn't contain any relevant information to the question, don't make something up and just say \"I don't know\":\n",
    "\n",
    "<context>\n",
    "{context}\n",
    "</context>\n",
    "\"\"\"\n",
    "\n",
    "question_answering_prompt = ChatPromptTemplate.from_messages(\n",
    "    [\n",
    "        (\n",
    "            \"system\",\n",
    "            SYSTEM_TEMPLATE,\n",
    "        ),\n",
    "        MessagesPlaceholder(variable_name=\"messages\"),\n",
    "    ]\n",
    ")\n",
    "\n",
    "document_chain = create_stuff_documents_chain(chat, question_answering_prompt)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "We can invoke this `document_chain` by itself to answer questions. Let's use the docs we retrieved above and the same question, `how can langsmith help with testing?`:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 8,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "'Yes, LangSmith can help test and evaluate your LLM applications. It simplifies the initial setup, and you can use it to monitor your application, log all traces, visualize latency and token usage statistics, and troubleshoot specific issues as they arise.'"
      ]
     },
     "execution_count": 8,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "from langchain_core.messages import HumanMessage\n",
    "\n",
    "document_chain.invoke(\n",
    "    {\n",
    "        \"context\": docs,\n",
    "        \"messages\": [\n",
    "            HumanMessage(content=\"Can LangSmith help test my LLM applications?\")\n",
    "        ],\n",
    "    }\n",
    ")"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Looks good! For comparison, we can try it with no context docs and compare the result:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 9,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "\"I don't know about LangSmith's specific capabilities for testing LLM applications. It's best to reach out to LangSmith directly to inquire about their services and how they can assist with testing your LLM applications.\""
      ]
     },
     "execution_count": 9,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "document_chain.invoke(\n",
    "    {\n",
    "        \"context\": [],\n",
    "        \"messages\": [\n",
    "            HumanMessage(content=\"Can LangSmith help test my LLM applications?\")\n",
    "        ],\n",
    "    }\n",
    ")"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "We can see that the LLM does not return any results.\n",
    "\n",
    "## Retrieval chains\n",
    "\n",
    "Let's combine this document chain with the retriever. Here's one way this can look:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 10,
   "metadata": {},
   "outputs": [],
   "source": [
    "from typing import Dict\n",
    "\n",
    "from langchain_core.runnables import RunnablePassthrough\n",
    "\n",
    "\n",
    "def parse_retriever_input(params: Dict):\n",
    "    return params[\"messages\"][-1].content\n",
    "\n",
    "\n",
    "retrieval_chain = RunnablePassthrough.assign(\n",
    "    context=parse_retriever_input | retriever,\n",
    ").assign(\n",
    "    answer=document_chain,\n",
    ")"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Given a list of input messages, we extract the content of the last message in the list and pass that to the retriever to fetch some documents. Then, we pass those documents as context to our document chain to generate a final response.\n",
    "\n",
    "Invoking this chain combines both steps outlined above:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 11,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "{'messages': [HumanMessage(content='Can LangSmith help test my LLM applications?')],\n",
       " 'context': [Document(page_content='Skip to main content🦜🛠 LangSmith DocsPython DocsJS/TS DocsSearchGo to AppLangSmithOverviewTracingTesting & EvaluationOrganizationsHubLangSmith CookbookOverviewOn this pageLangSmith Overview and User GuideBuilding reliable LLM applications can be challenging. LangChain simplifies the initial setup, but there is still work needed to bring the performance of prompts, chains and agents up the level where they are reliable enough to be used in production.Over the past two months, we at LangChain', metadata={'description': 'Building reliable LLM applications can be challenging. LangChain simplifies the initial setup, but there is still work needed to bring the performance of prompts, chains and agents up the level where they are reliable enough to be used in production.', 'language': 'en', 'source': 'https://docs.smith.langchain.com/overview', 'title': 'LangSmith Overview and User Guide | 🦜🛠 LangSmith'}),\n",
       "  Document(page_content='LangSmith Overview and User Guide | 🦜🛠 LangSmith', metadata={'description': 'Building reliable LLM applications can be challenging. LangChain simplifies the initial setup, but there is still work needed to bring the performance of prompts, chains and agents up the level where they are reliable enough to be used in production.', 'language': 'en', 'source': 'https://docs.smith.langchain.com/overview', 'title': 'LangSmith Overview and User Guide | 🦜🛠 LangSmith'}),\n",
       "  Document(page_content='You can also quickly edit examples and add them to datasets to expand the surface area of your evaluation sets or to fine-tune a model for improved quality or reduced costs.Monitoring\\u200bAfter all this, your app might finally ready to go in production. LangSmith can also be used to monitor your application in much the same way that you used for debugging. You can log all traces, visualize latency and token usage statistics, and troubleshoot specific issues as they arise. Each run can also be', metadata={'description': 'Building reliable LLM applications can be challenging. LangChain simplifies the initial setup, but there is still work needed to bring the performance of prompts, chains and agents up the level where they are reliable enough to be used in production.', 'language': 'en', 'source': 'https://docs.smith.langchain.com/overview', 'title': 'LangSmith Overview and User Guide | 🦜🛠 LangSmith'}),\n",


"  Document(page_content=\"does that affect the output?\\u200bSo you notice a bad output, and you go into LangSmith to see what's going on. You find the faulty LLM call and are now looking at the exact input. You want to try changing a word or a phrase to see what happens -- what do you do?We constantly ran into this issue. Initially, we copied the prompt to a playground of sorts. But this got annoying, so we built a playground of our own! When examining an LLM call, you can click the Open in Playground button to access this\", metadata={'description': 'Building reliable LLM applications can be challenging. LangChain simplifies the initial setup, but there is still work needed to bring the performance of prompts, chains and agents up the level where they are reliable enough to be used in production.', 'language': 'en', 'source': 'https://docs.smith.langchain.com/overview', 'title': 'LangSmith Overview and User Guide | 🦜🛠 LangSmith'})],\n",
       " 'answer': 'Yes, LangSmith can help test and evaluate your LLM applications. It simplifies the initial setup, and you can use it to monitor your application, log all traces, visualize latency and token usage statistics, and troubleshoot specific issues as they arise.'}"
      ]
     },
     "execution_count": 11,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "retrieval_chain.invoke(\n",
    "    {\n",
    "        \"messages\": [\n",
    "            HumanMessage(content=\"Can LangSmith help test my LLM applications?\")\n",
    "        ],\n",
    "    }\n",
    ")"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Looks good!\n",
    "\n",
    "## Query transformation\n",
    "\n",
    "Our retrieval chain is capable of answering questions about LangSmith, but there's a problem - chatbots interact with users conversationally, and therefore have to deal with followup questions.\n",
    "\n",
    "The chain in its current form will struggle with this. Consider a followup question to our original question like `Tell me more!`. If we invoke our retriever with that query directly, we get documents irrelevant to LLM application testing:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 12,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "[Document(page_content='You can also quickly edit examples and add them to datasets to expand the surface area of your evaluation sets or to fine-tune a model for improved quality or reduced costs.Monitoring\\u200bAfter all this, your app might finally ready to go in production. LangSmith can also be used to monitor your application in much the same way that you used for debugging. You can log all traces, visualize latency and token usage statistics, and troubleshoot specific issues as they arise. Each run can also be', metadata={'description': 'Building reliable LLM applications can be challenging. LangChain simplifies the initial setup, but there is still work needed to bring the performance of prompts, chains and agents up the level where they are reliable enough to be used in production.', 'language': 'en', 'source': 'https://docs.smith.langchain.com/overview', 'title': 'LangSmith Overview and User Guide | 🦜🛠 LangSmith'}),\n",
       " Document(page_content='playground. Here, you can modify the prompt and re-run it to observe the resulting changes to the output - as many times as needed!Currently, this feature supports only OpenAI and Anthropic models and works for LLM and Chat Model calls. We plan to extend its functionality to more LLM types, chains, agents, and retrievers in the future.What is the exact sequence of events?\\u200bIn complicated chains and agents, it can often be hard to understand what is going on under the hood. What calls are being', metadata={'description': 'Building reliable LLM applications can be challenging. LangChain simplifies the initial setup, but there is still work needed to bring the performance of prompts, chains and agents up the level where they are reliable enough to be used in production.', 'language': 'en', 'source': 'https://docs.smith.langchain.com/overview', 'title': 'LangSmith Overview and User Guide | 🦜🛠 LangSmith'}),\n",
       " Document(page_content='however, there is still no complete substitute for human review to get the utmost quality and reliability from your application.', metadata={'description': 'Building reliable LLM applications can be challenging. LangChain simplifies the initial setup, but there is still work needed to bring the performance of prompts, chains and agents up the level where they are reliable enough to be used in production.', 'language': 'en', 'source': 'https://docs.smith.langchain.com/overview', 'title': 'LangSmith Overview and User Guide | 🦜🛠 LangSmith'}),\n",
       " Document(page_content='Skip to main content🦜🛠 LangSmith DocsPython DocsJS/TS DocsSearchGo to AppLangSmithOverviewTracingTesting & EvaluationOrganizationsHubLangSmith CookbookOverviewOn this pageLangSmith Overview and User GuideBuilding reliable LLM applications can be challenging. LangChain simplifies the initial setup, but there is still work needed to bring the performance of prompts, chains and agents up the level where they are reliable enough to be used in production.Over the past two months, we at LangChain', metadata={'description': 'Building reliable LLM applications can be challenging. LangChain simplifies the initial setup, but there is still work needed to bring the performance of prompts, chains and agents up the level where they are reliable enough to be used in production.', 'language': 'en', 'source': 'https://docs.smith.langchain.com/overview', 'title': 'LangSmith Overview and User Guide | 🦜🛠 LangSmith'})]"
      ]
     },
     "execution_count": 12,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "retriever.invoke(\"Tell me more!\")"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "This is because the retriever has no innate concept of state, and will only pull documents most similar to the query given. To solve this, we can transform the query into a standalone query without any external references an LLM.\n",
    "\n",
    "Here's an example:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 13,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "AIMessage(content='\"LangSmith LLM application testing and evaluation\"')"
      ]
     },
     "execution_count": 13,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "from langchain_core.messages import AIMessage, HumanMessage\n",
    "\n",
    "query_transform_prompt = ChatPromptTemplate.from_messages(\n",
    "    [\n",
    "        MessagesPlaceholder(variable_name=\"messages\"),\n",
    "        (\n",
    "            \"user\",\n",
    "            \"Given the above conversation, generate a search query to look up in order to get information relevant to the conversation. Only respond with the query, nothing else.\",\n",
    "        ),\n",
    "    ]\n",
    ")\n",
    "\n",
    "query_transformation_chain = query_transform_prompt | chat\n",
    "\n",
    "query_transformation_chain.invoke(\n",
    "    {\n",
    "        \"messages\": [\n",
    "            HumanMessage(content=\"Can LangSmith help test my LLM applications?\"),\n",
    "            AIMessage(\n",
    "                content=\"Yes, LangSmith can help test and evaluate your LLM applications. It allows you to quickly edit examples and add them to datasets to expand the surface area of your evaluation sets or to fine-tune a model for improved quality or reduced costs. Additionally, LangSmith can be used to monitor your application, log all traces, visualize latency and token usage statistics, and troubleshoot specific issues as they arise.\"\n",
    "            ),\n",
    "            HumanMessage(content=\"Tell me more!\"),\n",
    "        ],\n",
    "    }\n",
    ")"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Awesome! That transformed query would pull up context documents related to LLM application testing.\n",
    "\n",
    "Let's add this to our retrieval chain. We can wrap our retriever as follows:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 14,
   "metadata": {},
   "outputs": [],
   "source": [
    "from langchain_core.output_parsers import StrOutputParser\n",
    "from langchain_core.runnables import RunnableBranch\n",
    "\n",