"For example, below we run inference on `llama2-13b` with 4 bit quantization downloaded from [HuggingFace](https://huggingface.co/TheBloke/Llama-2-13B-GGML/tree/main).\n",
    "\n",
    "As noted above, see the [API reference](https://python.langchain.com/api_reference/langchain/llms/langchain.llms.llamacpp.LlamaCpp.html?highlight=llamacpp#langchain.llms.llamacpp.LlamaCpp) for the full set of parameters. \n",
    "\n",
    "From the [llama.cpp API reference docs](https://python.langchain.com/api_reference/community/llms/langchain_community.llms.llamacpp.LlamaCpp.html), a few are worth commenting on:\n",
    "\n",
    "`n_gpu_layers`: number of layers to be loaded into GPU memory\n",
    "\n",
    "* Value: 1\n",
    "* Meaning: Only one layer of the model will be loaded into GPU memory (1 is often sufficient).\n",
    "\n",
    "`n_batch`: number of tokens the model should process in parallel \n",
    "\n",
    "* Value: n_batch\n",
    "* Meaning: It's recommended to choose a value between 1 and n_ctx (which in this case is set to 2048)\n",
    "\n",
    "`n_ctx`: Token context window\n",
    "\n",
    "* Value: 2048\n",
    "* Meaning: The model will consider a window of 2048 tokens at a time\n",
    "\n",
    "`f16_kv`: whether the model should use half-precision for the key/value cache\n",
    "\n",
    "* Value: True\n",
    "* Meaning: The model will use half-precision, which can be more memory efficient; Metal only supports True."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "5eba38dc",
   "metadata": {},
   "outputs": [],
   "source": [
    "%env CMAKE_ARGS=\"-DLLAMA_METAL=on\"\n",
    "%env FORCE_CMAKE=1\n",
    "%pip install --upgrade --quiet  llama-cpp-python --no-cache-dirclear"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "a88bf0c8-e989-4bcd-bcb7-4d7757e684f2",
   "metadata": {},
   "outputs": [],
   "source": [
    "from langchain_community.llms import LlamaCpp\n",
    "from langchain_core.callbacks import CallbackManager, StreamingStdOutCallbackHandler\n",
    "\n",
    "llm = LlamaCpp(\n",
    "    model_path=\"/Users/rlm/Desktop/Code/llama.cpp/models/openorca-platypus2-13b.gguf.q4_0.bin\",\n",
    "    n_gpu_layers=1,\n",
    "    n_batch=512,\n",
    "    n_ctx=2048,\n",
    "    f16_kv=True,\n",
    "    callback_manager=CallbackManager([StreamingStdOutCallbackHandler()]),\n",
    "    verbose=True,\n",
    ")"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "f56f5168",
   "metadata": {},
   "source": [
    "The console log will show the below to indicate Metal was enabled properly from steps above:\n",
    "```\n",
    "ggml_metal_init: allocating\n",
    "ggml_metal_init: using MPS\n",
    "```"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 45,
   "id": "7890a077",
   "metadata": {},
   "outputs": [
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "Llama.generate: prefix-match hit\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      " and use logical reasoning to figure out who the first man on the moon was.\n",
      "\n",
      "Here are some clues:\n",
      "\n",
      "1. The first man on the moon was an American.\n",
      "2. He was part of the Apollo 11 mission.\n",
      "3. He stepped out of the lunar module and became the first person to set foot on the moon's surface.\n",
      "4. His last name is Armstrong.\n",
      "\n",
      "Now, let's use our reasoning skills to figure out who the first man on the moon was. Based on clue #1, we know that the first man on the moon was an American. Clue #2 tells us that he was part of the Apollo 11 mission. Clue #3 reveals that he was the first person to set foot on the moon's surface. And finally, clue #4 gives us his last name: Armstrong.\n",
      "Therefore, the first man on the moon was Neil Armstrong!"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "\n",
      "llama_print_timings:        load time =  9623.21 ms\n",
      "llama_print_timings:      sample time =   143.77 ms /   203 runs   (    0.71 ms per token,  1412.01 tokens per second)\n",
      "llama_print_timings: prompt eval time =   485.94 ms /     7 tokens (   69.42 ms per token,    14.40 tokens per second)\n",
      "llama_print_timings:        eval time =  6385.16 ms /   202 runs   (   31.61 ms per token,    31.64 tokens per second)\n",
      "llama_print_timings:       total time =  7279.28 ms\n"
     ]
    },
    {
     "data": {
      "text/plain": [
       "\" and use logical reasoning to figure out who the first man on the moon was.\\n\\nHere are some clues:\\n\\n1. The first man on the moon was an American.\\n2. He was part of the Apollo 11 mission.\\n3. He stepped out of the lunar module and became the first person to set foot on the moon's surface.\\n4. His last name is Armstrong.\\n\\nNow, let's use our reasoning skills to figure out who the first man on the moon was. Based on clue #1, we know that the first man on the moon was an American. Clue #2 tells us that he was part of the Apollo 11 mission. Clue #3 reveals that he was the first person to set foot on the moon's surface. And finally, clue #4 gives us his last name: Armstrong.\\nTherefore, the first man on the moon was Neil Armstrong!\""
      ]
     },
     "execution_count": 45,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "llm.invoke(\"The first man on the moon was ... Let's think step by step\")"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "831ddf7c",
   "metadata": {},
   "source": [
    "### GPT4All\n",
    "\n",
    "We can use model weights downloaded from [GPT4All](/docs/integrations/llms/gpt4all) model explorer.\n",
    "\n",
    "Similar to what is shown above, we can run inference and use [the API reference](https://python.langchain.com/api_reference/community/llms/langchain_community.llms.gpt4all.GPT4All.html) to set parameters of interest."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "e27baf6e",
   "metadata": {},
   "outputs": [],
   "source": [
    "%pip install gpt4all"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "915ecd4c-8f6b-4de3-a787-b64cb7c682b4",
   "metadata": {},
   "outputs": [],
   "source": [
    "from langchain_community.llms import GPT4All\n",
    "\n",
    "llm = GPT4All(\n",
    "    model=\"/Users/rlm/Desktop/Code/gpt4all/models/nous-hermes-13b.ggmlv3.q4_0.bin\"\n",
    ")"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 47,
   "id": "e3d4526f",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [


{
 "cells": [
  {
   "cell_type": "raw",
   "metadata": {
    "vscode": {
     "languageId": "raw"
    }
   },
   "source": [
    "---\n",
    "keywords: [Runnable, Runnables, RunnableSequence, LCEL, chain, chains, chaining]\n",
    "---"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# How to chain runnables\n",
    "\n",
    ":::info Prerequisites\n",
    "\n",
    "This guide assumes familiarity with the following concepts:\n",
    "- [LangChain Expression Language (LCEL)](/docs/concepts/#langchain-expression-language)\n",
    "- [Prompt templates](/docs/concepts/#prompt-templates)\n",
    "- [Chat models](/docs/concepts/#chat-models)\n",
    "- [Output parser](/docs/concepts/#output-parsers)\n",
    "\n",
    ":::\n",
    "\n",
    "One point about [LangChain Expression Language](/docs/concepts/#langchain-expression-language) is that any two runnables can be \"chained\" together into sequences. The output of the previous runnable's `.invoke()` call is passed as input to the next runnable. This can be done using the pipe operator (`|`), or the more explicit `.pipe()` method, which does the same thing.\n",
    "\n",
    "The resulting [`RunnableSequence`](https://python.langchain.com/api_reference/core/runnables/langchain_core.runnables.base.RunnableSequence.html) is itself a runnable, which means it can be invoked, streamed, or further chained just like any other runnable. Advantages of chaining runnables in this way are efficient streaming (the sequence will stream output as soon as it is available), and debugging and tracing with tools like [LangSmith](/docs/how_to/debugging).\n",
    "\n",
    "## The pipe operator: `|`\n",
    "\n",
    "To show off how this works, let's go through an example. We'll walk through a common pattern in LangChain: using a [prompt template](/docs/how_to#prompt-templates) to format input into a [chat model](/docs/how_to#chat-models), and finally converting the chat message output into a string with an [output parser](/docs/how_to#output-parsers).\n",
    "\n",
    "import ChatModelTabs from \"@theme/ChatModelTabs\";\n",
    "\n",
    "<ChatModelTabs\n",
    "  customVarName=\"model\"\n",
    "/>\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 1,
   "metadata": {},
   "outputs": [],
   "source": [
    "# | output: false\n",
    "# | echo: false\n",
    "\n",
    "%pip install -qU langchain langchain_anthropic\n",
    "\n",
    "import os\n",
    "from getpass import getpass\n",
    "\n",
    "from langchain_anthropic import ChatAnthropic\n",
    "\n",
    "if \"ANTHROPIC_API_KEY\" not in os.environ:\n",
    "    os.environ[\"ANTHROPIC_API_KEY\"] = getpass()\n",
    "\n",
    "model = ChatAnthropic(model=\"claude-3-sonnet-20240229\", temperature=0)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 2,
   "metadata": {},
   "outputs": [],
   "source": [
    "from langchain_core.output_parsers import StrOutputParser\n",
    "from langchain_core.prompts import ChatPromptTemplate\n",
    "\n",
    "prompt = ChatPromptTemplate.from_template(\"tell me a joke about {topic}\")\n",
    "\n",
    "chain = prompt | model | StrOutputParser()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Prompts and models are both runnable, and the output type from the prompt call is the same as the input type of the chat model, so we can chain them together. We can then invoke the resulting sequence like any other runnable:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 3,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "\"Here's a bear joke for you:\\n\\nWhy did the bear dissolve in water?\\nBecause it was a polar bear!\""
      ]
     },
     "execution_count": 3,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "chain.invoke({\"topic\": \"bears\"})"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Coercion\n",
    "\n",
    "We can even combine this chain with more runnables to create another chain. This may involve some input/output formatting using other types of runnables, depending on the required inputs and outputs of the chain components.\n",
    "\n",
    "For example, let's say we wanted to compose the joke generating chain with another chain that evaluates whether or not the generated joke was funny.\n",
    "\n",
    "We would need to be careful with how we format the input into the next chain. In the below example, the dict in the chain is automatically parsed and converted into a [`RunnableParallel`](/docs/how_to/parallel), which runs all of its values in parallel and returns a dict with the results.\n",
    "\n",
    "This happens to be the same format the next prompt template expects. Here it is in action:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 4,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "'Haha, that\\'s a clever play on words! Using \"polar\" to imply the bear dissolved or became polar/polarized when put in water. Not the most hilarious joke ever, but it has a cute, groan-worthy pun that makes it mildly amusing. I appreciate a good pun or wordplay joke.'"
      ]
     },
     "execution_count": 4,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "from langchain_core.output_parsers import StrOutputParser\n",
    "\n",
    "analysis_prompt = ChatPromptTemplate.from_template(\"is this a funny joke? {joke}\")\n",
    "\n",
    "composed_chain = {\"joke\": chain} | analysis_prompt | model | StrOutputParser()\n",
    "\n",
    "composed_chain.invoke({\"topic\": \"bears\"})"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Functions will also be coerced into runnables, so you can add custom logic to your chains too. The below chain results in the same logical flow as before:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 5,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "\"Haha, that's a cute and punny joke! I like how it plays on the idea of beets blushing or turning red like someone blushing. Food puns can be quite amusing. While not a total knee-slapper, it's a light-hearted, groan-worthy dad joke that would make me chuckle and shake my head. Simple vegetable humor!\""
      ]
     },
     "execution_count": 5,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "composed_chain_with_lambda = (\n",
    "    chain\n",
    "    | (lambda input: {\"joke\": input})\n",
    "    | analysis_prompt\n",
    "    | model\n",
    "    | StrOutputParser()\n",
    ")\n",
    "\n",
    "composed_chain_with_lambda.invoke({\"topic\": \"beets\"})"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "However, keep in mind that using functions like this may interfere with operations like streaming. See [this section](/docs/how_to/functions) for more information."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## The `.pipe()` method\n",
    "\n",
    "We could also compose the same sequence using the `.pipe()` method. Here's what that looks like:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 6,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [


{
 "cells": [
  {
   "cell_type": "markdown",
   "id": "674a0d41-e3e3-4423-a995-25d40128c518",
   "metadata": {},
   "source": [
    "# How to do question answering over CSVs\n",
    "\n",
    "LLMs are great for building question-answering systems over various types of data sources. In this section we'll go over how to build Q&A systems over data stored in a CSV file(s). Like working with SQL databases, the key to working with CSV files is to give an LLM access to tools for querying and interacting with the data. The two main ways to do this are to either:\n",
    "\n",
    "* **RECOMMENDED**: Load the CSV(s) into a SQL database, and use the approaches outlined in the [SQL tutorial](/docs/tutorials/sql_qa).\n",
    "* Give the LLM access to a Python environment where it can use libraries like Pandas to interact with the data.\n",
    "\n",
    "We will cover both approaches in this guide.\n",
    "\n",
    "## ⚠ Security note ⚠\n",
    "\n",
    "Both approaches mentioned above carry significant risks. Using SQL requires executing model-generated SQL queries. Using a library like Pandas requires letting the model execute Python code. Since it is easier to tightly scope SQL connection permissions and sanitize SQL queries than it is to sandbox Python environments, **we HIGHLY recommend interacting with CSV data via SQL.** For more on general security best practices, [see here](/docs/security)."
   ]
  },
  {
   "cell_type": "markdown",
   "id": "d20c20d7-71e1-4808-9012-48278f3a9b94",
   "metadata": {},
   "source": [
    "## Setup\n",
    "Dependencies for this guide:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "c3fcf245-b0aa-4aee-8f0a-9c9cf94b065e",
   "metadata": {},
   "outputs": [],
   "source": [
    "%pip install -qU langchain langchain-openai langchain-community langchain-experimental pandas"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "7f2e34a3-0978-4856-8844-d8dfc6d5ac51",
   "metadata": {},
   "source": [
    "Set required environment variables:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 1,
   "id": "53913d79-4a11-4bc6-bb49-dea2cc8c453b",
   "metadata": {},
   "outputs": [],
   "source": [
    "# Using LangSmith is recommended but not required. Uncomment below lines to use.\n",
    "# import os\n",
    "# os.environ[\"LANGCHAIN_TRACING_V2\"] = \"true\"\n",
    "# os.environ[\"LANGCHAIN_API_KEY\"] = getpass.getpass()"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "c23b4232-2f6a-4eb5-b0cb-1d48a9e02fcc",
   "metadata": {},
   "source": [
    "Download the [Titanic dataset](https://www.kaggle.com/datasets/yasserh/titanic-dataset) if you don't already have it:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "1c9099c7-5247-4edb-ba5d-10c3c4c60db4",
   "metadata": {},
   "outputs": [],
   "source": [
    "!wget https://web.stanford.edu/class/archive/cs/cs109/cs109.1166/stuff/titanic.csv -O titanic.csv"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 1,
   "id": "ad029641-6d6c-44cc-b16f-2d5472672adf",
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "(887, 8)\n",
      "['Survived', 'Pclass', 'Name', 'Sex', 'Age', 'Siblings/Spouses Aboard', 'Parents/Children Aboard', 'Fare']\n"
     ]
    }
   ],
   "source": [
    "import pandas as pd\n",
    "\n",
    "df = pd.read_csv(\"titanic.csv\")\n",
    "print(df.shape)\n",
    "print(df.columns.tolist())"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "1779ab07-b715-49e5-ab2a-2e6be7d02927",
   "metadata": {},
   "source": [
    "## SQL\n",
    "\n",
    "Using SQL to interact with CSV data is the recommended approach because it is easier to limit permissions and sanitize queries than with arbitrary Python.\n",
    "\n",
    "Most SQL databases make it easy to load a CSV file in as a table ([DuckDB](https://duckdb.org/docs/data/csv/overview.html), [SQLite](https://www.sqlite.org/csv.html), etc.). Once you've done this you can use all of the chain and agent-creating techniques outlined in the [SQL tutorial](/docs/tutorials/sql_qa). Here's a quick example of how we might do this with SQLite:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 2,
   "id": "f61e9886-4713-4c88-87d4-dab439687f43",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "887"
      ]
     },
     "execution_count": 2,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "from langchain_community.utilities import SQLDatabase\n",
    "from sqlalchemy import create_engine\n",
    "\n",
    "engine = create_engine(\"sqlite:///titanic.db\")\n",
    "df.to_sql(\"titanic\", engine, index=False)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 3,
   "id": "3275fc91-3777-4f78-8edf-d148001684b0",
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "sqlite\n",
      "['titanic']\n",
      "[(1, 2, 'Master. Alden Gates Caldwell', 'male', 0.83, 0, 2, 29.0), (0, 3, 'Master. Eino Viljami Panula', 'male', 1.0, 4, 1, 39.6875), (1, 3, 'Miss. Eleanor Ileen Johnson', 'female', 1.0, 1, 1, 11.1333), (1, 2, 'Master. Richard F Becker', 'male', 1.0, 2, 1, 39.0), (1, 1, 'Master. Hudson Trevor Allison', 'male', 0.92, 1, 2, 151.55), (1, 3, 'Miss. Maria Nakid', 'female', 1.0, 0, 2, 15.7417), (0, 3, 'Master. Sidney Leonard Goodwin', 'male', 1.0, 5, 2, 46.9), (1, 3, 'Miss. Helene Barbara Baclini', 'female', 0.75, 2, 1, 19.2583), (1, 3, 'Miss. Eugenie Baclini', 'female', 0.75, 2, 1, 19.2583), (1, 2, 'Master. Viljo Hamalainen', 'male', 0.67, 1, 1, 14.5), (1, 3, 'Master. Bertram Vere Dean', 'male', 1.0, 1, 2, 20.575), (1, 3, 'Master. Assad Alexander Thomas', 'male', 0.42, 0, 1, 8.5167), (1, 2, 'Master. Andre Mallet', 'male', 1.0, 0, 2, 37.0042), (1, 2, 'Master. George Sibley Richards', 'male', 0.83, 1, 1, 18.75)]\n"
     ]
    }
   ],
   "source": [
    "db = SQLDatabase(engine=engine)\n",


"print(db.dialect)\n",
    "print(db.get_usable_table_names())\n",
    "print(db.run(\"SELECT * FROM titanic WHERE Age < 2;\"))"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "42f5a3c3-707c-4331-9f5f-0cb4919763dd",
   "metadata": {},
   "source": [
    "And create a [SQL agent](/docs/tutorials/sql_qa) to interact with it:\n",
    "\n",
    "import ChatModelTabs from \"@theme/ChatModelTabs\";\n",
    "\n",
    "<ChatModelTabs customVarName=\"llm\" />\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 6,
   "id": "e868a586-4f4e-4b1d-ab11-fae1271dd551",
   "metadata": {},
   "outputs": [],
   "source": [
    "# | output: false\n",
    "# | echo: false\n",
    "\n",
    "from langchain_openai import ChatOpenAI\n",
    "\n",
    "llm = ChatOpenAI()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 7,
   "id": "edd92649-b178-47bd-b2b7-d5d4e14b3512",
   "metadata": {},
   "outputs": [],
   "source": [
    "from langchain_community.agent_toolkits import create_sql_agent\n",
    "\n",
    "agent_executor = create_sql_agent(llm, db=db, agent_type=\"openai-tools\", verbose=True)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 8,
   "id": "7aefe929-5e39-4ed1-b135-aaf88edce2eb",
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "\n",
      "\n",
      "\u001b[1m> Entering new SQL Agent Executor chain...\u001b[0m\n",
      "\u001b[32;1m\u001b[1;3m\n",
      "Invoking: `sql_db_list_tables` with `{}`\n",
      "\n",
      "\n",
      "\u001b[0m\u001b[38;5;200m\u001b[1;3mtitanic\u001b[0m\u001b[32;1m\u001b[1;3m\n",
      "Invoking: `sql_db_schema` with `{'table_names': 'titanic'}`\n",
      "\n",
      "\n",
      "\u001b[0m\u001b[33;1m\u001b[1;3m\n",
      "CREATE TABLE titanic (\n",
      "\t\"Survived\" BIGINT, \n",
      "\t\"Pclass\" BIGINT, \n",
      "\t\"Name\" TEXT, \n",
      "\t\"Sex\" TEXT, \n",
      "\t\"Age\" FLOAT, \n",
      "\t\"Siblings/Spouses Aboard\" BIGINT, \n",
      "\t\"Parents/Children Aboard\" BIGINT, \n",
      "\t\"Fare\" FLOAT\n",
      ")\n",
      "\n",
      "/*\n",
      "3 rows from titanic table:\n",
      "Survived\tPclass\tName\tSex\tAge\tSiblings/Spouses Aboard\tParents/Children Aboard\tFare\n",
      "0\t3\tMr. Owen Harris Braund\tmale\t22.0\t1\t0\t7.25\n",
      "1\t1\tMrs. John Bradley (Florence Briggs Thayer) Cumings\tfemale\t38.0\t1\t0\t71.2833\n",
      "1\t3\tMiss. Laina Heikkinen\tfemale\t26.0\t0\t0\t7.925\n",
      "*/\u001b[0m\u001b[32;1m\u001b[1;3m\n",
      "Invoking: `sql_db_query` with `{'query': 'SELECT AVG(Age) AS Average_Age FROM titanic WHERE Survived = 1'}`\n",
      "\n",
      "\n",
      "\u001b[0m\u001b[36;1m\u001b[1;3m[(28.408391812865496,)]\u001b[0m\u001b[32;1m\u001b[1;3mThe average age of survivors in the Titanic dataset is approximately 28.41 years.\u001b[0m\n",
      "\n",
      "\u001b[1m> Finished chain.\u001b[0m\n"
     ]
    },
    {
     "data": {
      "text/plain": [
       "{'input': \"what's the average age of survivors\",\n",
       " 'output': 'The average age of survivors in the Titanic dataset is approximately 28.41 years.'}"
      ]
     },
     "execution_count": 8,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "agent_executor.invoke({\"input\": \"what's the average age of survivors\"})"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "4d1eb128-842b-4018-87ab-bb269147f6ec",
   "metadata": {},
   "source": [
    "This approach easily generalizes to multiple CSVs, since we can just load each of them into our database as its own table. See the [Multiple CSVs](/docs/how_to/sql_csv#multiple-csvs) section below."
   ]
  },
  {
   "cell_type": "markdown",
   "id": "fe7f2d91-2377-49dd-97a3-19d48a750715",
   "metadata": {},
   "source": [
    "## Pandas\n",
    "\n",
    "Instead of SQL we can also use data analysis libraries like pandas and the code generating abilities of LLMs to interact with CSV data. Again, **this approach is not fit for production use cases unless you have extensive safeguards in place**. For this reason, our code-execution utilities and constructors live in the `langchain-experimental` package.\n",
    "\n",
    "### Chain\n",
    "\n",
    "Most LLMs have been trained on enough pandas Python code that they can generate it just by being asked to:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 9,
   "id": "27c84b27-9367-4c58-8a88-ade1fbf6683c",
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "```python\n",
      "correlation = df['Age'].corr(df['Fare'])\n",
      "correlation\n",
      "```\n"
     ]
    }
   ],
   "source": [
    "ai_msg = llm.invoke(\n",
    "    \"I have a pandas DataFrame 'df' with columns 'Age' and 'Fare'. Write code to compute the correlation between the two columns. Return Markdown for a Python code snippet and nothing else.\"\n",
    ")\n",
    "print(ai_msg.content)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "f5e84003-5c39-496b-afa7-eaa50a01b7bb",
   "metadata": {},
   "source": [
    "We can combine this ability with a Python-executing tool to create a simple data analysis chain. We'll first want to load our CSV table as a dataframe, and give the tool access to this dataframe:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 10,
   "id": "16abe312-b1a3-413f-bb9a-0e613d1e550b",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "32.30542018038331"
      ]
     },
     "execution_count": 10,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "import pandas as pd\n",
    "from langchain_core.prompts import ChatPromptTemplate\n",
    "from langchain_experimental.tools import PythonAstREPLTool\n",
    "\n",
    "df = pd.read_csv(\"titanic.csv\")\n",
    "tool = PythonAstREPLTool(locals={\"df\": df})\n",
    "tool.invoke(\"df['Fare'].mean()\")"
   ]
  },
  {
   "cell_type": "markdown",


"    [\n",
    "        (\n",
    "            \"system\",\n",
    "            system,\n",
    "        ),\n",
    "        (\"human\", \"{question}\"),\n",
    "        # This MessagesPlaceholder allows us to optionally append an arbitrary number of messages\n",
    "        # at the end of the prompt using the 'chat_history' arg.\n",
    "        MessagesPlaceholder(\"chat_history\", optional=True),\n",
    "    ]\n",
    ")\n",
    "\n",
    "\n",
    "def _get_chat_history(x: dict) -> list:\n",
    "    \"\"\"Parse the chain output up to this point into a list of chat history messages to insert in the prompt.\"\"\"\n",
    "    ai_msg = x[\"ai_msg\"]\n",
    "    tool_call_id = x[\"ai_msg\"].additional_kwargs[\"tool_calls\"][0][\"id\"]\n",
    "    tool_msg = ToolMessage(tool_call_id=tool_call_id, content=str(x[\"tool_output\"]))\n",
    "    return [ai_msg, tool_msg]\n",
    "\n",
    "\n",
    "chain = (\n",
    "    RunnablePassthrough.assign(ai_msg=prompt | llm_with_tools)\n",
    "    .assign(tool_output=itemgetter(\"ai_msg\") | parser | tool)\n",
    "    .assign(chat_history=_get_chat_history)\n",
    "    .assign(response=prompt | llm | StrOutputParser())\n",
    "    .pick([\"tool_output\", \"response\"])\n",
    ")"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 17,
   "id": "ff6e98ec-52f1-4ffd-9ea8-bacedfa29f28",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "{'tool_output': 0.11232863699941616,\n",
       " 'response': 'The correlation between age and fare is approximately 0.1123.'}"
      ]
     },
     "execution_count": 17,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "chain.invoke({\"question\": \"What's the correlation between age and fare\"})"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "245a5a91-c6d2-4a40-9b9f-eb38f78c9d22",
   "metadata": {},
   "source": [
    "Here's the LangSmith trace for this run: https://smith.langchain.com/public/14e38d70-45b1-4b81-8477-9fd2b7c07ea6/r"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "6c24b4f4-abbf-4891-b200-814eb9c35bec",
   "metadata": {},
   "source": [
    "### Agent\n",
    "\n",
    "For complex questions it can be helpful for an LLM to be able to iteratively execute code while maintaining the inputs and outputs of its previous executions. This is where Agents come into play. They allow an LLM to decide how many times a tool needs to be invoked and keep track of the executions it's made so far. The [create_pandas_dataframe_agent](https://python.langchain.com/api_reference/experimental/agents/langchain_experimental.agents.agent_toolkits.pandas.base.create_pandas_dataframe_agent.html) is a built-in agent that makes it easy to work with dataframes:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 18,
   "id": "35ea904e-795f-411b-bef8-6484dbb6e35c",
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "\n",
      "\n",
      "\u001b[1m> Entering new AgentExecutor chain...\u001b[0m\n",
      "\u001b[32;1m\u001b[1;3m\n",
      "Invoking: `python_repl_ast` with `{'query': \"df[['Age', 'Fare']].corr().iloc[0,1]\"}`\n",
      "\n",
      "\n",
      "\u001b[0m\u001b[36;1m\u001b[1;3m0.11232863699941621\u001b[0m\u001b[32;1m\u001b[1;3m\n",
      "Invoking: `python_repl_ast` with `{'query': \"df[['Fare', 'Survived']].corr().iloc[0,1]\"}`\n",
      "\n",
      "\n",
      "\u001b[0m\u001b[36;1m\u001b[1;3m0.2561785496289603\u001b[0m\u001b[32;1m\u001b[1;3mThe correlation between Age and Fare is approximately 0.112, and the correlation between Fare and Survival is approximately 0.256.\n",
      "\n",
      "Therefore, the correlation between Fare and Survival (0.256) is greater than the correlation between Age and Fare (0.112).\u001b[0m\n",
      "\n",
      "\u001b[1m> Finished chain.\u001b[0m\n"
     ]
    },
    {
     "data": {
      "text/plain": [
       "{'input': \"What's the correlation between age and fare? is that greater than the correlation between fare and survival?\",\n",
       " 'output': 'The correlation between Age and Fare is approximately 0.112, and the correlation between Fare and Survival is approximately 0.256.\\n\\nTherefore, the correlation between Fare and Survival (0.256) is greater than the correlation between Age and Fare (0.112).'}"
      ]
     },
     "execution_count": 18,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "from langchain_experimental.agents import create_pandas_dataframe_agent\n",
    "\n",
    "agent = create_pandas_dataframe_agent(llm, df, agent_type=\"openai-tools\", verbose=True)\n",
    "agent.invoke(\n",
    "    {\n",
    "        \"input\": \"What's the correlation between age and fare? is that greater than the correlation between fare and survival?\"\n",
    "    }\n",
    ")"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "a65322f3-b13c-4949-82b2-4517b9a0859d",
   "metadata": {},
   "source": [
    "Here's the LangSmith trace for this run: https://smith.langchain.com/public/6a86aee2-4f22-474a-9264-bd4c7283e665/r"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "68492261-faef-47e7-8009-e20ef1420d5a",
   "metadata": {},
   "source": [
    "### Multiple CSVs {#multiple-csvs}\n",
    "\n",
    "To handle multiple CSVs (or dataframes) we just need to pass multiple dataframes to our Python tool. Our `create_pandas_dataframe_agent` constructor can do this out of the box, we can pass in a list of dataframes instead of just one. If we're constructing a chain ourselves, we can do something like:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 19,
   "id": "77a70e1b-d3ee-4fa6-a4a0-d2e5005e6c8a",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "0.14384991262954416"
      ]
     },
     "execution_count": 19,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "df_1 = df[[\"Age\", \"Fare\"]]\n",
    "df_2 = df[[\"Fare\", \"Survived\"]]\n",
    "\n",
    "tool = PythonAstREPLTool(locals={\"df_1\": df_1, \"df_2\": df_2})\n",
    "llm_with_tool = llm.bind_tools(tools=[tool], tool_choice=tool.name)\n",
    "df_template = \"\"\"```python\n",
    "{df_name}.head().to_markdown()\n",
    ">>> {df_head}\n",
    "```\"\"\"\n",
    "df_context = \"\\n\\n\".join(\n",


"    df_template.format(df_head=_df.head().to_markdown(), df_name=df_name)\n",
    "    for _df, df_name in [(df_1, \"df_1\"), (df_2, \"df_2\")]\n",
    ")\n",
    "\n",
    "system = f\"\"\"You have access to a number of pandas dataframes. \\\n",
    "Here is a sample of rows from each dataframe and the python code that was used to generate the sample:\n",
    "\n",
    "{df_context}\n",
    "\n",
    "Given a user question about the dataframes, write the Python code to answer it. \\\n",
    "Don't assume you have access to any libraries other than built-in Python ones and pandas. \\\n",
    "Make sure to refer only to the variables mentioned above.\"\"\"\n",
    "prompt = ChatPromptTemplate.from_messages([(\"system\", system), (\"human\", \"{question}\")])\n",
    "\n",
    "chain = prompt | llm_with_tool | parser | tool\n",
    "chain.invoke(\n",
    "    {\n",
    "        \"question\": \"return the difference in the correlation between age and fare and the correlation between fare and survival\"\n",
    "    }\n",
    ")"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "7043363f-4ab1-41de-9318-c556e4ae66bc",
   "metadata": {},
   "source": [
    "Here's the LangSmith trace for this run: https://smith.langchain.com/public/cc2a7d7f-7c5a-4e77-a10c-7b5420fcd07f/r"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "a2256d09-23c2-4e52-bfc6-c84eba538586",
   "metadata": {},
   "source": [
    "### Sandboxed code execution\n",
    "\n",
    "There are a number of tools like [E2B](/docs/integrations/tools/e2b_data_analysis) and [Bearly](/docs/integrations/tools/bearly) that provide sandboxed environments for Python code execution, to allow for safer code-executing chains and agents."
   ]
  },
  {
   "cell_type": "markdown",
   "id": "1728e791-f114-41e6-aa12-0436fdeeedae",
   "metadata": {},
   "source": [
    "## Next steps\n",
    "\n",
    "For more advanced data analysis applications we recommend checking out:\n",
    "\n",
    "* [SQL tutorial](/docs/tutorials/sql_qa): Many of the challenges of working with SQL db's and CSV's are generic to any structured data type, so it's useful to read the SQL techniques even if you're using Pandas for CSV data analysis.\n",
    "* [Tool use](/docs/how_to/tool_calling): Guides on general best practices when working with chains and agents that invoke tools\n",
    "* [Agents](/docs/tutorials/agents): Understand the fundamentals of building LLM agents.\n",
    "* Integrations: Sandboxed envs like [E2B](/docs/integrations/tools/e2b_data_analysis) and [Bearly](/docs/integrations/tools/bearly), utilities like [SQLDatabase](https://python.langchain.com/api_reference/community/utilities/langchain_community.utilities.sql_database.SQLDatabase.html#langchain_community.utilities.sql_database.SQLDatabase), related agents like [Spark DataFrame agent](/docs/integrations/tools/spark_sql)."
   ]
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 3 (ipykernel)",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.10.4"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 5
}


---
sidebar_position: 5
keywords: [RunnablePassthrough, LCEL]
---

# How to pass through arguments from one step to the next

This guide assumes familiarity with the following concepts:

- [LangChain Expression Language (LCEL)](/docs/concepts/#langchain-expression-language)

- [Calling runnables in parallel](/docs/how_to/parallel/)

When composing chains with several steps, sometimes you will want to pass data from previous steps unchanged for use as input to a later step. The [`RunnablePassthrough`](https://python.langchain.com/api_reference/core/runnables/langchain_core.runnables.passthrough.RunnablePassthrough.html) class allows you to do just this, and is typically is used in conjuction with a [RunnableParallel](/docs/how_to/parallel/) to pass data through to a later step in your constructed chains.

%pip install -qU langchain langchain-openai

from langchain_core.runnables import RunnableParallel, RunnablePassthrough

As seen above, `passed` key was called with `RunnablePassthrough()` and so it simply passed on `{'num': 1}`.

We also set a second key in the map with `modified`. This uses a lambda to set a single value adding 1 to the num, which resulted in `modified` key with the value of `2`.

In the example below, we see a more real-world use case where we use `RunnablePassthrough` along with `RunnableParallel` in a chain to properly format inputs to a prompt:

from langchain_community.vectorstores import FAISS

from langchain_core.output_parsers import StrOutputParser

from langchain_core.prompts import ChatPromptTemplate

from langchain_core.runnables import RunnablePassthrough

from langchain_openai import ChatOpenAI, OpenAIEmbeddings

["harrison worked at kensho"], embedding=OpenAIEmbeddings()

template = """Answer the question based only on the following context:

{"context": retriever, "question": RunnablePassthrough()}

retrieval_chain.invoke("where did harrison work?")

Here the input to prompt is expected to be a map with keys "context" and "question". The user input is just the question. So we need to get the context using our retriever and passthrough the user input under the "question" key. The `RunnablePassthrough` allows us to pass on the user's question to the prompt and model.

Now you've learned how to pass data through your chains to help to help format the data flowing through your chains.

To learn more, see the other how-to guides on runnables in this section.


This guide assumes familiarity with the following concepts:

- [How to use a model to call tools](/docs/how_to/tool_calling)

In order to force our LLM to select a specific tool, we can use the `tool_choice` parameter to ensure certain behavior. First, let's define our model and tools:

from langchain_core.tools import tool
@tool
def add(a: int, b: int) -> int:
"""Adds a and b."""
return a + b
@tool
def multiply(a: int, b: int) -> int:
"""Multiplies a and b."""
return a * b
tools = [add, multiply]

%pip install -qU langchain langchain_openai

llm = ChatOpenAI(model="gpt-4o-mini", temperature=0)

For example, we can force our tool to call the multiply tool by using the following code:

AIMessage(content='', additional_kwargs={'tool_calls': [{'id': 'call_9cViskmLvPnHjXk9tbVla5HA', 'function': {'arguments': '{"a":2,"b":4}', 'name': 'Multiply'}, 'type': 'function'}]}, response_metadata={'token_usage': {'completion_tokens': 9, 'prompt_tokens': 103, 'total_tokens': 112}, 'model_name': 'gpt-4o-mini', 'system_fingerprint': None, 'finish_reason': 'stop', 'logprobs': None}, id='run-095b827e-2bdd-43bb-8897-c843f4504883-0', tool_calls=[{'name': 'Multiply', 'args': {'a': 2, 'b': 4}, 'id': 'call_9cViskmLvPnHjXk9tbVla5HA'}], usage_metadata={'input_tokens': 103, 'output_tokens': 9, 'total_tokens': 112})

llm_forced_to_multiply = llm.bind_tools(tools, tool_choice="Multiply")

llm_forced_to_multiply.invoke("what is 2 + 4")

Even if we pass it something that doesn't require multiplcation - it will still call the tool!

We can also just force our tool to select at least one of our tools by passing in the "any" (or "required" which is OpenAI specific) keyword to the `tool_choice` parameter.

AIMessage(content='', additional_kwargs={'tool_calls': [{'id': 'call_mCSiJntCwHJUBfaHZVUB2D8W', 'function': {'arguments': '{"a":1,"b":2}', 'name': 'Add'}, 'type': 'function'}]}, response_metadata={'token_usage': {'completion_tokens': 15, 'prompt_tokens': 94, 'total_tokens': 109}, 'model_name': 'gpt-4o-mini', 'system_fingerprint': None, 'finish_reason': 'stop', 'logprobs': None}, id='run-28f75260-9900-4bed-8cd3-f1579abb65e5-0', tool_calls=[{'name': 'Add', 'args': {'a': 1, 'b': 2}, 'id': 'call_mCSiJntCwHJUBfaHZVUB2D8W'}], usage_metadata={'input_tokens': 94, 'output_tokens': 15, 'total_tokens': 109})

llm_forced_to_use_tool = llm.bind_tools(tools, tool_choice="any")

llm_forced_to_use_tool.invoke("What day is today?")


{
 "cells": [
  {
   "cell_type": "raw",
   "id": "94c3ad61",
   "metadata": {},
   "source": [
    "---\n",
    "sidebar_position: 3\n",
    "---"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "b91e03f1",
   "metadata": {},
   "source": [
    "# How to use few shot examples\n",
    "\n",
    ":::info Prerequisites\n",
    "\n",
    "This guide assumes familiarity with the following concepts:\n",
    "- [Prompt templates](/docs/concepts/#prompt-templates)\n",
    "- [Example selectors](/docs/concepts/#example-selectors)\n",
    "- [LLMs](/docs/concepts/#llms)\n",
    "- [Vectorstores](/docs/concepts/#vector-stores)\n",
    "\n",
    ":::\n",
    "\n",
    "In this guide, we'll learn how to create a simple prompt template that provides the model with example inputs and outputs when generating. Providing the LLM with a few such examples is called few-shotting, and is a simple yet powerful way to guide generation and in some cases drastically improve model performance.\n",
    "\n",
    "A few-shot prompt template can be constructed from either a set of examples, or from an [Example Selector](https://python.langchain.com/api_reference/core/example_selectors/langchain_core.example_selectors.base.BaseExampleSelector.html) class responsible for choosing a subset of examples from the defined set.\n",
    "\n",
    "This guide will cover few-shotting with string prompt templates. For a guide on few-shotting with chat messages for chat models, see [here](/docs/how_to/few_shot_examples_chat/).\n",
    "\n",
    "## Create a formatter for the few-shot examples\n",
    "\n",
    "Configure a formatter that will format the few-shot examples into a string. This formatter should be a `PromptTemplate` object."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 1,
   "id": "4e70bce2",
   "metadata": {},
   "outputs": [],
   "source": [
    "from langchain_core.prompts import PromptTemplate\n",
    "\n",
    "example_prompt = PromptTemplate.from_template(\"Question: {question}\\n{answer}\")"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "50846ad4",
   "metadata": {},
   "source": [
    "## Creating the example set\n",
    "\n",
    "Next, we'll create a list of few-shot examples. Each example should be a dictionary representing an example input to the formatter prompt we defined above."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 2,
   "id": "a44be840",
   "metadata": {},
   "outputs": [],
   "source": [
    "examples = [\n",
    "    {\n",
    "        \"question\": \"Who lived longer, Muhammad Ali or Alan Turing?\",\n",
    "        \"answer\": \"\"\"\n",
    "Are follow up questions needed here: Yes.\n",
    "Follow up: How old was Muhammad Ali when he died?\n",
    "Intermediate answer: Muhammad Ali was 74 years old when he died.\n",
    "Follow up: How old was Alan Turing when he died?\n",
    "Intermediate answer: Alan Turing was 41 years old when he died.\n",
    "So the final answer is: Muhammad Ali\n",
    "\"\"\",\n",
    "    },\n",
    "    {\n",
    "        \"question\": \"When was the founder of craigslist born?\",\n",
    "        \"answer\": \"\"\"\n",
    "Are follow up questions needed here: Yes.\n",
    "Follow up: Who was the founder of craigslist?\n",
    "Intermediate answer: Craigslist was founded by Craig Newmark.\n",
    "Follow up: When was Craig Newmark born?\n",
    "Intermediate answer: Craig Newmark was born on December 6, 1952.\n",
    "So the final answer is: December 6, 1952\n",
    "\"\"\",\n",
    "    },\n",
    "    {\n",
    "        \"question\": \"Who was the maternal grandfather of George Washington?\",\n",
    "        \"answer\": \"\"\"\n",
    "Are follow up questions needed here: Yes.\n",
    "Follow up: Who was the mother of George Washington?\n",
    "Intermediate answer: The mother of George Washington was Mary Ball Washington.\n",
    "Follow up: Who was the father of Mary Ball Washington?\n",
    "Intermediate answer: The father of Mary Ball Washington was Joseph Ball.\n",
    "So the final answer is: Joseph Ball\n",
    "\"\"\",\n",
    "    },\n",
    "    {\n",
    "        \"question\": \"Are both the directors of Jaws and Casino Royale from the same country?\",\n",
    "        \"answer\": \"\"\"\n",
    "Are follow up questions needed here: Yes.\n",
    "Follow up: Who is the director of Jaws?\n",
    "Intermediate Answer: The director of Jaws is Steven Spielberg.\n",
    "Follow up: Where is Steven Spielberg from?\n",
    "Intermediate Answer: The United States.\n",
    "Follow up: Who is the director of Casino Royale?\n",
    "Intermediate Answer: The director of Casino Royale is Martin Campbell.\n",
    "Follow up: Where is Martin Campbell from?\n",
    "Intermediate Answer: New Zealand.\n",
    "So the final answer is: No\n",
    "\"\"\",\n",
    "    },\n",
    "]"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "3d1ec9d5",
   "metadata": {},
   "source": [
    "Let's test the formatting prompt with one of our examples:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 13,
   "id": "8c6e48ad",
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Question: Who lived longer, Muhammad Ali or Alan Turing?\n",
      "\n",
      "Are follow up questions needed here: Yes.\n",
      "Follow up: How old was Muhammad Ali when he died?\n",
      "Intermediate answer: Muhammad Ali was 74 years old when he died.\n",
      "Follow up: How old was Alan Turing when he died?\n",
      "Intermediate answer: Alan Turing was 41 years old when he died.\n",
      "So the final answer is: Muhammad Ali\n",
      "\n"
     ]
    }
   ],
   "source": [
    "print(example_prompt.invoke(examples[0]).to_string())"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "dad66af1",
   "metadata": {},
   "source": [
    "### Pass the examples and formatter to `FewShotPromptTemplate`\n",
    "\n",
    "Finally, create a [`FewShotPromptTemplate`](https://python.langchain.com/api_reference/core/prompts/langchain_core.prompts.few_shot.FewShotPromptTemplate.html) object. This object takes in the few-shot examples and the formatter for the few-shot examples. When this `FewShotPromptTemplate` is formatted, it formats the passed examples using the `example_prompt`, then and adds them to the final prompt before `suffix`:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 14,
   "id": "e76fa1ba",
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Question: Who lived longer, Muhammad Ali or Alan Turing?\n",
      "\n",
      "Are follow up questions needed here: Yes.\n",
      "Follow up: How old was Muhammad Ali when he died?\n",
      "Intermediate answer: Muhammad Ali was 74 years old when he died.\n",
      "Follow up: How old was Alan Turing when he died?\n",
      "Intermediate answer: Alan Turing was 41 years old when he died.\n",
      "So the final answer is: Muhammad Ali\n",
      "\n",
      "\n",
      "Question: When was the founder of craigslist born?\n",
      "\n",
      "Are follow up questions needed here: Yes.\n",
      "Follow up: Who was the founder of craigslist?\n",
      "Intermediate answer: Craigslist was founded by Craig Newmark.\n",
      "Follow up: When was Craig Newmark born?\n",
      "Intermediate answer: Craig Newmark was born on December 6, 1952.\n",
      "So the final answer is: December 6, 1952\n",
      "\n",
      "\n",
      "Question: Who was the maternal grandfather of George Washington?\n",
      "\n",
      "Are follow up questions needed here: Yes.\n",


# How to use a vectorstore as a retriever

A vector store retriever is a retriever that uses a vector store to retrieve documents. It is a lightweight wrapper around the vector store class to make it conform to the retriever interface.

It uses the search methods implemented by a vector store, like similarity search and MMR, to query the texts in the vector store.

1. How to instantiate a retriever from a vectorstore;

2. How to specify the search type for the retriever;

3. How to specify additional search parameters, such as threshold scores and top-k.

## Creating a retriever from a vectorstore

You can build a retriever from a vectorstore using its [.as_retriever](https://python.langchain.com/api_reference/core/vectorstores/langchain_core.vectorstores.VectorStore.html#langchain_core.vectorstores.VectorStore.as_retriever) method. Let's walk through an example.

First we instantiate a vectorstore. We will use an in-memory [FAISS](https://python.langchain.com/api_reference/community/vectorstores/langchain_community.vectorstores.faiss.FAISS.html) vectorstore:

from langchain_community.document_loaders import TextLoader

from langchain_community.vectorstores import FAISS

from langchain_openai import OpenAIEmbeddings

from langchain_text_splitters import CharacterTextSplitter

text_splitter = CharacterTextSplitter(chunk_size=1000, chunk_overlap=0)

vectorstore = FAISS.from_documents(texts, embeddings)

This creates a retriever (specifically a [VectorStoreRetriever](https://python.langchain.com/api_reference/core/vectorstores/langchain_core.vectorstores.base.VectorStoreRetriever.html)), which we can use in the usual way:

docs = retriever.invoke("what did the president say about ketanji brown jackson?")

By default, the vector store retriever uses similarity search. If the underlying vector store supports maximum marginal relevance search, you can specify that as the search type.

This effectively specifies what method on the underlying vectorstore is used (e.g., `similarity_search`, `max_marginal_relevance_search`, etc.).

We can pass parameters to the underlying vectorstore's search methods using `search_kwargs`.

### Similarity score threshold retrieval

For example, we can set a similarity score threshold and only return documents with a score above that threshold.

We can also limit the number of documents `k` returned by the retriever.

retriever = vectorstore.as_retriever(search_kwargs={"k": 1})


{
 "cells": [
  {
   "cell_type": "markdown",
   "id": "9122e4b9-4883-4e6e-940b-ab44a70f0951",
   "metadata": {},
   "source": [
    "# How to load documents from a directory\n",
    "\n",
    "LangChain's [DirectoryLoader](https://python.langchain.com/api_reference/community/document_loaders/langchain_community.document_loaders.directory.DirectoryLoader.html) implements functionality for reading files from disk into LangChain [Document](https://python.langchain.com/api_reference/core/documents/langchain_core.documents.base.Document.html#langchain_core.documents.base.Document) objects. Here we demonstrate:\n",
    "\n",
    "- How to load from a filesystem, including use of wildcard patterns;\n",
    "- How to use multithreading for file I/O;\n",
    "- How to use custom loader classes to parse specific file types (e.g., code);\n",
    "- How to handle errors, such as those due to decoding."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 1,
   "id": "1c1e3796-bee8-4882-8065-6b98e48ec53a",
   "metadata": {},
   "outputs": [],
   "source": [
    "from langchain_community.document_loaders import DirectoryLoader"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "e3cdb7bb-1f58-4a7a-af83-599443127834",
   "metadata": {},
   "source": [
    "`DirectoryLoader` accepts a `loader_cls` kwarg, which defaults to [UnstructuredLoader](/docs/integrations/document_loaders/unstructured_file). [Unstructured](https://unstructured-io.github.io/unstructured/) supports parsing for a number of formats, such as PDF and HTML. Here we use it to read in a markdown (.md) file.\n",
    "\n",
    "We can use the `glob` parameter to control which files to load. Note that here it doesn't load the `.rst` file or the `.html` files."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 2,
   "id": "bd2fcd1f-8286-499b-b43a-0c17084ae8ee",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "20"
      ]
     },
     "execution_count": 2,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "loader = DirectoryLoader(\"../\", glob=\"**/*.md\")\n",
    "docs = loader.load()\n",
    "len(docs)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 3,
   "id": "9ff1503d-3ac0-4172-99ec-15c9a4a707d8",
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Security\n",
      "\n",
      "LangChain has a large ecosystem of integrations with various external resources like local\n"
     ]
    }
   ],
   "source": [
    "print(docs[0].page_content[:100])"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "b8b1cee8-626a-461a-8d33-1c56120f1cc0",
   "metadata": {},
   "source": [
    "## Show a progress bar\n",
    "\n",
    "By default a progress bar will not be shown. To show a progress bar, install the `tqdm` library (e.g. `pip install tqdm`), and set the `show_progress` parameter to `True`."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 4,
   "id": "cfa48224-5d02-4aa7-93c7-ce48241645d5",
   "metadata": {},
   "outputs": [
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "100%|█████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████| 20/20 [00:00<00:00, 54.56it/s]\n"
     ]
    }
   ],
   "source": [
    "loader = DirectoryLoader(\"../\", glob=\"**/*.md\", show_progress=True)\n",
    "docs = loader.load()"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "5e02c922-6a4b-48e6-8c46-5015553eafbe",
   "metadata": {},
   "source": [
    "## Use multithreading\n",
    "\n",
    "By default the loading happens in one thread. In order to utilize several threads set the `use_multithreading` flag to true."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 5,
   "id": "aae1c580-6d7c-409c-bfc8-3049fa8bdbf9",
   "metadata": {},
   "outputs": [],
   "source": [
    "loader = DirectoryLoader(\"../\", glob=\"**/*.md\", use_multithreading=True)\n",
    "docs = loader.load()"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "5add3f54-f303-4006-90c9-540a90ab8c46",
   "metadata": {},
   "source": [
    "## Change loader class\n",
    "By default this uses the `UnstructuredLoader` class. To customize the loader, specify the loader class in the `loader_cls` kwarg. Below we show an example using [TextLoader](https://python.langchain.com/api_reference/community/document_loaders/langchain_community.document_loaders.text.TextLoader.html):"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 6,
   "id": "d369ee78-ea24-48cc-9f46-1f5cd4b56f48",
   "metadata": {},
   "outputs": [],
   "source": [
    "from langchain_community.document_loaders import TextLoader\n",
    "\n",
    "loader = DirectoryLoader(\"../\", glob=\"**/*.md\", loader_cls=TextLoader)\n",
    "docs = loader.load()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 7,
   "id": "2863d7dd-2d56-4fef-8bfd-95c48a6b4a71",
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "# Security\n",
      "\n",
      "LangChain has a large ecosystem of integrations with various external resources like loc\n"
     ]
    }
   ],
   "source": [
    "print(docs[0].page_content[:100])"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "c97ed37b-38c0-4f31-9403-d3a5d5444f78",
   "metadata": {},
   "source": [
    "Notice that while the `UnstructuredLoader` parses Markdown headers, `TextLoader` does not.\n",
    "\n",
    "If you need to load Python source code files, use the `PythonLoader`:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 8,
   "id": "5ef483a8-57d3-45e5-93be-37c8416c543c",
   "metadata": {},
   "outputs": [],
   "source": [
    "from langchain_community.document_loaders import PythonLoader\n",
    "\n",
    "loader = DirectoryLoader(\"../../../../../\", glob=\"**/*.py\", loader_cls=PythonLoader)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "61dd1428-8246-47e3-b1da-f6a3d6f05566",
   "metadata": {},
   "source": [
    "## Auto-detect file encodings with TextLoader\n",
    "\n",
    "`DirectoryLoader` can help manage errors due to variations in file encodings. Below we will attempt to load in a collection of files, one of which includes non-UTF8 encodings."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 9,
   "id": "e69db7ae-0385-4129-968f-17c42c7a635c",
   "metadata": {},
   "outputs": [],


keywords: [RunnablePassthrough, assign, LCEL]

This guide assumes familiarity with the following concepts:

- [LangChain Expression Language (LCEL)](/docs/concepts/#langchain-expression-language)

- [Calling runnables in parallel](/docs/how_to/parallel/)

- [Passing data through](/docs/how_to/passthrough)

An alternate way of [passing data through](/docs/how_to/passthrough) steps of a chain is to leave the current values of the chain state unchanged while assigning a new value under a given key. The [`RunnablePassthrough.assign()`](https://python.langchain.com/api_reference/core/runnables/langchain_core.runnables.passthrough.RunnablePassthrough.html#langchain_core.runnables.passthrough.RunnablePassthrough.assign) static method takes an input value and adds the extra arguments passed to the assign function.

This is useful in the common [LangChain Expression Language](/docs/concepts/#langchain-expression-language) pattern of additively creating a dictionary to use as input to a later step.

%pip install --upgrade --quiet langchain langchain-openai

from langchain_core.runnables import RunnableParallel, RunnablePassthrough

extra=RunnablePassthrough.assign(mult=lambda x: x["num"] * 3),

- The input to the chain is `{"num": 1}`. This is passed into a `RunnableParallel`, which invokes the runnables it is passed in parallel with that input.

- The value under the `extra` key is invoked. `RunnablePassthrough.assign()` keeps the original keys in the input dict (`{"num": 1}`), and assigns a new key called `mult`. The value is `lambda x: x["num"] * 3)`, which is `3`. Thus, the result is `{"num": 1, "mult": 3}`.

- `{"num": 1, "mult": 3}` is returned to the `RunnableParallel` call, and is set as the value to the key `extra`.

- At the same time, the `modified` key is called. The result is `2`, since the lambda extracts a key called `"num"` from its input and adds one.

Thus, the result is `{'extra': {'num': 1, 'mult': 3}, 'modified': 2}`.

One convenient feature of this method is that it allows values to pass through as soon as they are available. To show this off, we'll use `RunnablePassthrough.assign()` to immediately return source docs in a retrieval chain:

stdout

{'question': 'where did harrison work?'}

{'context': [Document(page_content='harrison worked at kensho')]}

{'output': ''}

{'output': 'H'}

{'output': 'arrison'}

{'output': ' worked'}

{'output': ' at'}

{'output': ' Kens'}

{'output': 'ho'}

{'output': '.'}

{'output': ''}

from langchain_community.vectorstores import FAISS

from langchain_core.output_parsers import StrOutputParser

from langchain_core.prompts import ChatPromptTemplate

from langchain_core.runnables import RunnablePassthrough

from langchain_openai import ChatOpenAI, OpenAIEmbeddings

["harrison worked at kensho"], embedding=OpenAIEmbeddings()

template = """Answer the question based only on the following context:

generation_chain = prompt | model | StrOutputParser()

stream = retrieval_chain.stream("where did harrison work?")

We can see that the first chunk contains the original `"question"` since that is immediately available. The second chunk contains `"context"` since the retriever finishes second. Finally, the output from the `generation_chain` streams in chunks as soon as it is available.

Now you've learned how to pass data through your chains to help to help format the data flowing through your chains.

To learn more, see the other how-to guides on runnables in this section.


"File \u001b[0;32m~/langchain/.venv/lib/python3.11/site-packages/langchain_core/tools/base.py:659\u001b[0m, in \u001b[0;36mBaseTool.run\u001b[0;34m(self, tool_input, verbose, start_color, color, callbacks, tags, metadata, run_name, run_id, config, tool_call_id, **kwargs)\u001b[0m\n\u001b[1;32m    657\u001b[0m \u001b[38;5;28;01mif\u001b[39;00m error_to_raise:\n\u001b[1;32m    658\u001b[0m     run_manager\u001b[38;5;241m.\u001b[39mon_tool_error(error_to_raise)\n\u001b[0;32m--> 659\u001b[0m     \u001b[38;5;28;01mraise\u001b[39;00m error_to_raise\n\u001b[1;32m    660\u001b[0m output \u001b[38;5;241m=\u001b[39m _format_output(content, artifact, tool_call_id, \u001b[38;5;28mself\u001b[39m\u001b[38;5;241m.\u001b[39mname, status)\n\u001b[1;32m    661\u001b[0m run_manager\u001b[38;5;241m.\u001b[39mon_tool_end(output, color\u001b[38;5;241m=\u001b[39mcolor, name\u001b[38;5;241m=\u001b[39m\u001b[38;5;28mself\u001b[39m\u001b[38;5;241m.\u001b[39mname, \u001b[38;5;241m*\u001b[39m\u001b[38;5;241m*\u001b[39mkwargs)\n",
      "File \u001b[0;32m~/langchain/.venv/lib/python3.11/site-packages/langchain_core/tools/base.py:622\u001b[0m, in \u001b[0;36mBaseTool.run\u001b[0;34m(self, tool_input, verbose, start_color, color, callbacks, tags, metadata, run_name, run_id, config, tool_call_id, **kwargs)\u001b[0m\n\u001b[1;32m    620\u001b[0m context \u001b[38;5;241m=\u001b[39m copy_context()\n\u001b[1;32m    621\u001b[0m context\u001b[38;5;241m.\u001b[39mrun(_set_config_context, child_config)\n\u001b[0;32m--> 622\u001b[0m tool_args, tool_kwargs \u001b[38;5;241m=\u001b[39m \u001b[38;5;28;43mself\u001b[39;49m\u001b[38;5;241;43m.\u001b[39;49m\u001b[43m_to_args_and_kwargs\u001b[49m\u001b[43m(\u001b[49m\u001b[43mtool_input\u001b[49m\u001b[43m)\u001b[49m\n\u001b[1;32m    623\u001b[0m \u001b[38;5;28;01mif\u001b[39;00m signature(\u001b[38;5;28mself\u001b[39m\u001b[38;5;241m.\u001b[39m_run)\u001b[38;5;241m.\u001b[39mparameters\u001b[38;5;241m.\u001b[39mget(\u001b[38;5;124m\"\u001b[39m\u001b[38;5;124mrun_manager\u001b[39m\u001b[38;5;124m\"\u001b[39m):\n\u001b[1;32m    624\u001b[0m     tool_kwargs[\u001b[38;5;124m\"\u001b[39m\u001b[38;5;124mrun_manager\u001b[39m\u001b[38;5;124m\"\u001b[39m] \u001b[38;5;241m=\u001b[39m run_manager\n",
      "File \u001b[0;32m~/langchain/.venv/lib/python3.11/site-packages/langchain_core/tools/base.py:545\u001b[0m, in \u001b[0;36mBaseTool._to_args_and_kwargs\u001b[0;34m(self, tool_input)\u001b[0m\n\u001b[1;32m    544\u001b[0m \u001b[38;5;28;01mdef\u001b[39;00m \u001b[38;5;21m_to_args_and_kwargs\u001b[39m(\u001b[38;5;28mself\u001b[39m, tool_input: Union[\u001b[38;5;28mstr\u001b[39m, Dict]) \u001b[38;5;241m-\u001b[39m\u001b[38;5;241m>\u001b[39m Tuple[Tuple, Dict]:\n\u001b[0;32m--> 545\u001b[0m     tool_input \u001b[38;5;241m=\u001b[39m \u001b[38;5;28;43mself\u001b[39;49m\u001b[38;5;241;43m.\u001b[39;49m\u001b[43m_parse_input\u001b[49m\u001b[43m(\u001b[49m\u001b[43mtool_input\u001b[49m\u001b[43m)\u001b[49m\n\u001b[1;32m    546\u001b[0m     \u001b[38;5;66;03m# For backwards compatibility, if run_input is a string,\u001b[39;00m\n\u001b[1;32m    547\u001b[0m     \u001b[38;5;66;03m# pass as a positional argument.\u001b[39;00m\n\u001b[1;32m    548\u001b[0m     \u001b[38;5;28;01mif\u001b[39;00m \u001b[38;5;28misinstance\u001b[39m(tool_input, \u001b[38;5;28mstr\u001b[39m):\n",


"\n",
    "ts_splitter = RecursiveCharacterTextSplitter.from_language(\n",
    "    language=Language.TS, chunk_size=60, chunk_overlap=0\n",
    ")\n",
    "ts_docs = ts_splitter.create_documents([TS_CODE])\n",
    "ts_docs"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "ee2361f8",
   "metadata": {},
   "source": [
    "## Markdown\n",
    "\n",
    "Here's an example using the Markdown text splitter:\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 2,
   "id": "ac9295d3",
   "metadata": {},
   "outputs": [],
   "source": [
    "markdown_text = \"\"\"\n",
    "# 🦜🔗 LangChain\n",
    "\n",
    "⚡ Building applications with LLMs through composability ⚡\n",
    "\n",
    "## Quick Install\n",
    "\n",
    "# Hopefully this code block isn't split\n",
    "pip install langchain\n",
    "\n",
    "As an open-source project in a rapidly developing field, we are extremely open to contributions.\n",
    "\"\"\""
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 3,
   "id": "3a0cb17a",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "[Document(page_content='# 🦜🔗 LangChain'),\n",
       " Document(page_content='⚡ Building applications with LLMs through composability ⚡'),\n",
       " Document(page_content='## Quick Install'),\n",
       " Document(page_content=\"# Hopefully this code block isn't split\"),\n",
       " Document(page_content='pip install langchain'),\n",
       " Document(page_content='As an open-source project in a rapidly developing field, we'),\n",
       " Document(page_content='are extremely open to contributions.')]"
      ]
     },
     "execution_count": 3,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "md_splitter = RecursiveCharacterTextSplitter.from_language(\n",
    "    language=Language.MARKDOWN, chunk_size=60, chunk_overlap=0\n",
    ")\n",
    "md_docs = md_splitter.create_documents([markdown_text])\n",
    "md_docs"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "7aa306f6",
   "metadata": {},
   "source": [
    "## Latex\n",
    "\n",
    "Here's an example on Latex text:\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 10,
   "id": "77d1049d",
   "metadata": {},
   "outputs": [],
   "source": [
    "latex_text = \"\"\"\n",
    "\\documentclass{article}\n",
    "\n",
    "\\begin{document}\n",
    "\n",
    "\\maketitle\n",
    "\n",
    "\\section{Introduction}\n",
    "Large language models (LLMs) are a type of machine learning model that can be trained on vast amounts of text data to generate human-like language. In recent years, LLMs have made significant advances in a variety of natural language processing tasks, including language translation, text generation, and sentiment analysis.\n",
    "\n",
    "\\subsection{History of LLMs}\n",
    "The earliest LLMs were developed in the 1980s and 1990s, but they were limited by the amount of data that could be processed and the computational power available at the time. In the past decade, however, advances in hardware and software have made it possible to train LLMs on massive datasets, leading to significant improvements in performance.\n",
    "\n",
    "\\subsection{Applications of LLMs}\n",
    "LLMs have many applications in industry, including chatbots, content creation, and virtual assistants. They can also be used in academia for research in linguistics, psychology, and computational linguistics.\n",
    "\n",
    "\\end{document}\n",
    "\"\"\""
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 11,
   "id": "4dbc47e1",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "[Document(page_content='\\\\documentclass{article}\\n\\n\\x08egin{document}\\n\\n\\\\maketitle'),\n",
       " Document(page_content='\\\\section{Introduction}'),\n",
       " Document(page_content='Large language models (LLMs) are a type of machine learning'),\n",
       " Document(page_content='model that can be trained on vast amounts of text data to'),\n",
       " Document(page_content='generate human-like language. In recent years, LLMs have'),\n",
       " Document(page_content='made significant advances in a variety of natural language'),\n",
       " Document(page_content='processing tasks, including language translation, text'),\n",
       " Document(page_content='generation, and sentiment analysis.'),\n",
       " Document(page_content='\\\\subsection{History of LLMs}'),\n",
       " Document(page_content='The earliest LLMs were developed in the 1980s and 1990s,'),\n",
       " Document(page_content='but they were limited by the amount of data that could be'),\n",
       " Document(page_content='processed and the computational power available at the'),\n",
       " Document(page_content='time. In the past decade, however, advances in hardware and'),\n",
       " Document(page_content='software have made it possible to train LLMs on massive'),\n",
       " Document(page_content='datasets, leading to significant improvements in'),\n",
       " Document(page_content='performance.'),\n",
       " Document(page_content='\\\\subsection{Applications of LLMs}'),\n",
       " Document(page_content='LLMs have many applications in industry, including'),\n",
       " Document(page_content='chatbots, content creation, and virtual assistants. They'),\n",
       " Document(page_content='can also be used in academia for research in linguistics,'),\n",
       " Document(page_content='psychology, and computational linguistics.'),\n",
       " Document(page_content='\\\\end{document}')]"
      ]
     },
     "execution_count": 11,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "latex_splitter = RecursiveCharacterTextSplitter.from_language(\n",
    "    language=Language.MARKDOWN, chunk_size=60, chunk_overlap=0\n",
    ")\n",
    "latex_docs = latex_splitter.create_documents([latex_text])\n",
    "latex_docs"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "c29adadf",
   "metadata": {},
   "source": [
    "## HTML\n",
    "\n",
    "Here's an example using an HTML text splitter:\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 12,
   "id": "0fc78794",
   "metadata": {},
   "outputs": [],
   "source": [
    "html_text = \"\"\"\n",
    "<!DOCTYPE html>\n",
    "\n",
    "    \n",
    "        <title>🦜🔗 LangChain</title>\n",
    "        \n",
    "    \n",
    "    \n",
    "
\n",
    "
🦜🔗 LangChain
\n",
    "
⚡ Building applications with LLMs through composability ⚡
\n",
    "
\n",
    "
\n",
    "            As an open-source project in a rapidly developing field, we are extremely open to contributions.\n",
    "
\n",
    "    \n",
    "\n",
    "\"\"\""
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 13,
   "id": "e3e3fca1",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "[Document(page_content='<!DOCTYPE html>\\n'),\n",
       " Document(page_content='\\n        <title>🦜🔗 LangChain</title>'),\n",


" Document(page_content='\\n    '),\n",
       " Document(page_content=''),\n",
       " Document(page_content='
\\n
🦜🔗 LangChain
'),\n",
       " Document(page_content='
⚡ Building applications with LLMs through composability ⚡'),\n",
       " Document(page_content='
\\n
'),\n",
       " Document(page_content='
\\n            As an open-source project in a rapidly dev'),\n",
       " Document(page_content='eloping field, we are extremely open to contributions.'),\n",
       " Document(page_content='
\\n    \\n')]"
      ]
     },
     "execution_count": 13,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "html_splitter = RecursiveCharacterTextSplitter.from_language(\n",
    "    language=Language.HTML, chunk_size=60, chunk_overlap=0\n",
    ")\n",
    "html_docs = html_splitter.create_documents([html_text])\n",
    "html_docs"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "fcaf7abf",
   "metadata": {},
   "source": [
    "## Solidity\n",
    "Here's an example using the Solidity text splitter:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 14,
   "id": "49a1df11",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "[Document(page_content='pragma solidity ^0.8.20;'),\n",
       " Document(page_content='contract HelloWorld {\\n   function add(uint a, uint b) pure public returns(uint) {\\n       return a + b;\\n   }\\n}')]"
      ]
     },
     "execution_count": 14,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "SOL_CODE = \"\"\"\n",
    "pragma solidity ^0.8.20;\n",
    "contract HelloWorld {\n",
    "   function add(uint a, uint b) pure public returns(uint) {\n",
    "       return a + b;\n",
    "   }\n",
    "}\n",
    "\"\"\"\n",
    "\n",
    "sol_splitter = RecursiveCharacterTextSplitter.from_language(\n",
    "    language=Language.SOL, chunk_size=128, chunk_overlap=0\n",
    ")\n",
    "sol_docs = sol_splitter.create_documents([SOL_CODE])\n",
    "sol_docs"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "edd0052c",
   "metadata": {},
   "source": [
    "## C#\n",
    "Here's an example using the C# text splitter:\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 15,
   "id": "1524ae0f",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "[Document(page_content='using System;'),\n",
       " Document(page_content='class Program\\n{\\n    static void Main()\\n    {\\n        int age = 30; // Change the age value as needed'),\n",
       " Document(page_content='// Categorize the age without any console output\\n        if (age < 18)\\n        {\\n            // Age is under 18'),\n",
       " Document(page_content='}\\n        else if (age >= 18 && age < 65)\\n        {\\n            // Age is an adult\\n        }\\n        else\\n        {'),\n",
       " Document(page_content='// Age is a senior citizen\\n        }\\n    }\\n}')]"
      ]
     },
     "execution_count": 15,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "C_CODE = \"\"\"\n",
    "using System;\n",
    "class Program\n",
    "{\n",
    "    static void Main()\n",
    "    {\n",
    "        int age = 30; // Change the age value as needed\n",
    "\n",
    "        // Categorize the age without any console output\n",
    "        if (age < 18)\n",
    "        {\n",
    "            // Age is under 18\n",
    "        }\n",
    "        else if (age >= 18 && age < 65)\n",
    "        {\n",
    "            // Age is an adult\n",
    "        }\n",
    "        else\n",
    "        {\n",
    "            // Age is a senior citizen\n",
    "        }\n",
    "    }\n",
    "}\n",
    "\"\"\"\n",
    "c_splitter = RecursiveCharacterTextSplitter.from_language(\n",
    "    language=Language.CSHARP, chunk_size=128, chunk_overlap=0\n",
    ")\n",
    "c_docs = c_splitter.create_documents([C_CODE])\n",
    "c_docs"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "af9de667-230e-4c2a-8c5f-122a28515d97",
   "metadata": {},
   "source": [
    "## Haskell\n",
    "Here's an example using the Haskell text splitter:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 3,
   "id": "688185b5",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "[Document(page_content='main :: IO ()'),\n",
       " Document(page_content='main = do\\n    putStrLn \"Hello, World!\"\\n-- Some'),\n",
       " Document(page_content='sample functions\\nadd :: Int -> Int -> Int\\nadd x y'),\n",
       " Document(page_content='= x + y')]"
      ]
     },
     "execution_count": 3,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "HASKELL_CODE = \"\"\"\n",
    "main :: IO ()\n",
    "main = do\n",
    "    putStrLn \"Hello, World!\"\n",
    "-- Some sample functions\n",
    "add :: Int -> Int -> Int\n",
    "add x y = x + y\n",
    "\"\"\"\n",
    "haskell_splitter = RecursiveCharacterTextSplitter.from_language(\n",
    "    language=Language.HASKELL, chunk_size=50, chunk_overlap=0\n",
    ")\n",
    "haskell_docs = haskell_splitter.create_documents([HASKELL_CODE])\n",
    "haskell_docs"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "4a11f7cd-cd85-430c-b307-5b5b5f07f8db",
   "metadata": {},
   "source": [
    "## PHP\n",
    "Here's an example using the PHP text splitter:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 2,
   "id": "90c66e7e-87a5-4a81-bece-7949aabf2369",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "[Document(page_content='<?php\\nnamespace foo;'),\n",
       " Document(page_content='class Hello {'),\n",
       " Document(page_content='public function __construct() { }\\n}'),\n",
       " Document(page_content='function hello() {\\n    echo \"Hello World!\";\\n}'),\n",
       " Document(page_content='interface Human {\\n    public function breath();\\n}'),\n",
       " Document(page_content='trait Foo { }\\nenum Color\\n{\\n    case Red;'),\n",
       " Document(page_content='case Blue;\\n}')]"
      ]
     },
     "execution_count": 2,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "PHP_CODE = \"\"\"<?php\n",
    "namespace foo;\n",
    "class Hello {\n",
    "    public function __construct() { }\n",
    "}\n",


{
 "cells": [
  {
   "cell_type": "markdown",
   "id": "7414502a-4532-4da3-aef0-71aac4d0d4dd",
   "metadata": {},
   "source": [
    "# How to load web pages\n",
    "\n",
    "This guide covers how to load web pages into the LangChain [Document](https://python.langchain.com/api_reference/core/documents/langchain_core.documents.base.Document.html) format that we use downstream. Web pages contain text, images, and other multimedia elements, and are typically represented with HTML. They may include links to other pages or resources.\n",
    "\n",
    "LangChain integrates with a host of parsers that are appropriate for web pages. The right parser will depend on your needs. Below we demonstrate two possibilities:\n",
    "\n",
    "- [Simple and fast](/docs/how_to/document_loader_web#simple-and-fast-text-extraction) parsing, in which we recover one `Document` per web page with its content represented as a \"flattened\" string;\n",
    "- [Advanced](/docs/how_to/document_loader_web#advanced-parsing) parsing, in which we recover multiple `Document` objects per page, allowing one to identify and traverse sections, links, tables, and other structures.\n",
    "\n",
    "## Setup\n",
    "\n",
    "For the \"simple and fast\" parsing, we will need `langchain-community` and the `beautifulsoup4` library:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "89bc7be9-ab50-4c5a-860a-deee7b469f67",
   "metadata": {},
   "outputs": [],
   "source": [
    "%pip install -qU langchain-community beautifulsoup4"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "a07f5ca3-e2b7-4d9c-b1f2-7547856cbdf7",
   "metadata": {},
   "source": [
    "For advanced parsing, we will use `langchain-unstructured`:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "8a3ef1fc-dfde-4814-b7f6-b6c0c649f044",
   "metadata": {},
   "outputs": [],
   "source": [
    "%pip install -qU langchain-unstructured"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "4ef11005-1bd0-43a3-8d52-ea823c830c34",
   "metadata": {},
   "source": [
    "## Simple and fast text extraction\n",
    "\n",
    "If you are looking for a simple string representation of text that is embedded in a web page, the method below is appropriate. It will return a list of `Document` objects -- one per page -- containing a single string of the page's text. Under the hood it uses the `beautifulsoup4` Python library.\n",
    "\n",
    "LangChain document loaders implement `lazy_load` and its async variant, `alazy_load`, which return iterators of `Document objects`. We will use these below."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 1,
   "id": "7faeccbc-4e56-4b88-99db-2274ed0680c1",
   "metadata": {},
   "outputs": [
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "USER_AGENT environment variable not set, consider setting it to identify your requests.\n"
     ]
    }
   ],
   "source": [
    "import bs4\n",
    "from langchain_community.document_loaders import WebBaseLoader\n",
    "\n",
    "page_url = \"https://python.langchain.com/docs/how_to/chatbots_memory/\"\n",
    "\n",
    "loader = WebBaseLoader(web_paths=[page_url])\n",
    "docs = []\n",
    "async for doc in loader.alazy_load():\n",
    "    docs.append(doc)\n",
    "\n",
    "assert len(docs) == 1\n",
    "doc = docs[0]"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 2,
   "id": "21199a0d-3bd2-4410-a060-763649b14691",
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "{'source': 'https://python.langchain.com/docs/how_to/chatbots_memory/', 'title': 'How to add memory to chatbots | \\uf8ffü¶úÔ∏è\\uf8ffüîó LangChain', 'description': 'A key feature of chatbots is their ability to use content of previous conversation turns as context. This state management can take several forms, including:', 'language': 'en'}\n",
      "\n",
      "How to add memory to chatbots | ü¶úÔ∏èüîó LangChain\n",
      "\n",
      "\n",
      "\n",
      "\n",
      "\n",
      "\n",
      "\n",
      "Skip to main contentShare your thoughts on AI agents. Take the 3-min survey.IntegrationsAPI ReferenceMoreContributingPeopleLangSmithLangGraphLangChain HubLangChain JS/TSv0.3v0.3v0.2v0.1üí¨SearchIntroductionTutorialsBuild a Question Answering application over a Graph DatabaseTutorialsBuild a Simple LLM Application with LCELBuild a Query Analysis SystemBuild a ChatbotConversational RAGBuild an Extraction ChainBuild an AgentTaggingd\n"
     ]
    }
   ],
   "source": [
    "print(f\"{doc.metadata}\\n\")\n",
    "print(doc.page_content[:500].strip())"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "23189e91-5237-4a9e-a4bb-cb79e130c364",
   "metadata": {},
   "source": [
    "This is essentially a dump of the text from the page's HTML. It may contain extraneous information like headings and navigation bars. If you are familiar with the expected HTML, you can specify desired `<div>` classes and other parameters via BeautifulSoup. Below we parse only the body text of the article:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 3,
   "id": "4211b1a6-e636-415b-a556-ae01969399a7",
   "metadata": {},
   "outputs": [],
   "source": [
    "loader = WebBaseLoader(\n",
    "    web_paths=[page_url],\n",
    "    bs_kwargs={\n",
    "        \"parse_only\": bs4.SoupStrainer(class_=\"theme-doc-markdown markdown\"),\n",
    "    },\n",
    "    bs_get_text_kwargs={\"separator\": \" | \", \"strip\": True},\n",
    ")\n",
    "\n",
    "docs = []\n",
    "async for doc in loader.alazy_load():\n",
    "    docs.append(doc)\n",
    "\n",
    "assert len(docs) == 1\n",
    "doc = docs[0]"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 4,
   "id": "7edf6ed0-e22f-4c64-b986-8ba019c14757",
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "{'source': 'https://python.langchain.com/docs/how_to/chatbots_memory/'}\n",
      "\n",
      "How to add memory to chatbots | A key feature of chatbots is their ability to use content of previous conversation turns as context. This state management can take several forms, including: | Simply stuffing previous messages into a chat model prompt. | The above, but trimming old messages to reduce the amount of distracting information the model has to deal with. | More complex modifications like synthesizing summaries for long running conversations. | We'll go into more detail on a few techniq\n"
     ]
    }
   ],
   "source": [
    "print(f\"{doc.metadata}\\n\")\n",
    "print(doc.page_content[:500])"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 5,
   "id": "6ab1ba2b-3b22-4c5d-8ad3-f6809d075d26",
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [


"{'https://python.langchain.com/docs/how_to/chatbots_memory/': \"You'll need to install a few packages, and have your OpenAI API key set as an environment variable named OPENAI_API_KEY:\\n%pip install --upgrade --quiet langchain langchain-openai\\n\\n# Set env var OPENAI_API_KEY or load from a .env file:\\nimport dotenv\\n\\ndotenv.load_dotenv()\\n[33mWARNING: You are using pip version 22.0.4; however, version 23.3.2 is available.\\nYou should consider upgrading via the '/Users/jacoblee/.pyenv/versions/3.10.5/bin/python -m pip install --upgrade pip' command.[0m[33m\\n[0mNote: you may need to restart the kernel to use updated packages.\\n\",\n",
       " 'https://python.langchain.com/docs/how_to/chatbots_tools/': \"For this guide, we'll be using a tool calling agent with a single tool for searching the web. The default will be powered by Tavily, but you can switch it out for any similar tool. The rest of this section will assume you're using Tavily.\\nYou'll need to sign up for an account on the Tavily website, and install the following packages:\\n%pip install --upgrade --quiet langchain-community langchain-openai tavily-python\\n\\n# Set env var OPENAI_API_KEY or load from a .env file:\\nimport dotenv\\n\\ndotenv.load_dotenv()\\nYou will also need your OpenAI key set as OPENAI_API_KEY and your Tavily API key set as TAVILY_API_KEY.\\n\"}"
      ]
     },
     "execution_count": 10,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "from collections import defaultdict\n",
    "\n",
    "setup_text = defaultdict(str)\n",
    "\n",
    "for doc in setup_docs:\n",
    "    url = doc.metadata[\"url\"]\n",
    "    setup_text[url] += f\"{doc.page_content}\\n\"\n",
    "\n",
    "dict(setup_text)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "5cd42892-24a6-4969-92c8-c928680be9b5",
   "metadata": {},
   "source": [
    "### Vector search over page content\n",
    "\n",
    "Once we have loaded the page contents into LangChain `Document` objects, we can index them (e.g., for a RAG application) in the usual way. Below we use OpenAI [embeddings](/docs/concepts/#embedding-models), although any LangChain embeddings model will suffice."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "5a6cbb01-6e0d-418f-9f76-2031622bebb0",
   "metadata": {},
   "outputs": [],
   "source": [
    "%pip install -qU langchain-openai"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 11,
   "id": "598e612c-180d-494d-8caa-761c89f84eae",
   "metadata": {},
   "outputs": [],
   "source": [
    "import getpass\n",
    "import os\n",
    "\n",
    "if \"OPENAI_API_KEY\" not in os.environ:\n",
    "    os.environ[\"OPENAI_API_KEY\"] = getpass.getpass(\"OpenAI API Key:\")"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 12,
   "id": "5eeaeb54-ea03-4634-8a79-b60c22ab2b66",
   "metadata": {},
   "outputs": [
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "INFO: HTTP Request: POST https://api.openai.com/v1/embeddings \"HTTP/1.1 200 OK\"\n",
      "INFO: HTTP Request: POST https://api.openai.com/v1/embeddings \"HTTP/1.1 200 OK\"\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Page https://python.langchain.com/docs/how_to/chatbots_tools/: You'll need to sign up for an account on the Tavily website, and install the following packages:\n",
      "\n",
      "Page https://python.langchain.com/docs/how_to/chatbots_tools/: For this guide, we'll be using a tool calling agent with a single tool for searching the web. The default will be powered by Tavily, but you can switch it out for any similar tool. The rest of this section will assume you're using Tavily.\n",
      "\n"
     ]
    }
   ],
   "source": [
    "from langchain_core.vectorstores import InMemoryVectorStore\n",
    "from langchain_openai import OpenAIEmbeddings\n",
    "\n",
    "vector_store = InMemoryVectorStore.from_documents(setup_docs, OpenAIEmbeddings())\n",
    "retrieved_docs = vector_store.similarity_search(\"Install Tavily\", k=2)\n",
    "for doc in retrieved_docs:\n",
    "    print(f'Page {doc.metadata[\"url\"]}: {doc.page_content[:300]}\\n')"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "67be9c94-dbde-4fdd-87d0-e83ed6066d2b",
   "metadata": {},
   "source": [
    "## Other web page loaders\n",
    "\n",
    "For a list of available LangChain web page loaders, please see [this table](/docs/integrations/document_loaders/#webpages)."
   ]
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 3 (ipykernel)",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.10.4"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 5
}


"id": "d59823f5-9b9a-43c5-a213-34644e2f1d3d",
   "metadata": {},
   "source": [
    ":::note\n",
    "Because the code above is relying on JSON auto-completion, you may see partial names of countries (e.g., `Sp` and `Spain`), which is not what one would want for an extraction result!\n",
    "\n",
    "We're focusing on streaming concepts, not necessarily the results of the chains.\n",
    ":::"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "6adf65b7-aa47-4321-98c7-a0abe43b833a",
   "metadata": {},
   "source": [
    "### Non-streaming components\n",
    "\n",
    "Some built-in components like Retrievers do not offer any `streaming`. What happens if we try to `stream` them? 🤨"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 10,
   "id": "b9b1c00d-8b44-40d0-9e2b-8a70d238f82b",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "[[Document(page_content='harrison worked at kensho'),\n",
       "  Document(page_content='harrison likes spicy food')]]"
      ]
     },
     "execution_count": 10,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "from langchain_community.vectorstores import FAISS\n",
    "from langchain_core.output_parsers import StrOutputParser\n",
    "from langchain_core.prompts import ChatPromptTemplate\n",
    "from langchain_core.runnables import RunnablePassthrough\n",
    "from langchain_openai import OpenAIEmbeddings\n",
    "\n",
    "template = \"\"\"Answer the question based only on the following context:\n",
    "{context}\n",
    "\n",
    "Question: {question}\n",
    "\"\"\"\n",
    "prompt = ChatPromptTemplate.from_template(template)\n",
    "\n",
    "vectorstore = FAISS.from_texts(\n",
    "    [\"harrison worked at kensho\", \"harrison likes spicy food\"],\n",
    "    embedding=OpenAIEmbeddings(),\n",
    ")\n",
    "retriever = vectorstore.as_retriever()\n",
    "\n",
    "chunks = [chunk for chunk in retriever.stream(\"where did harrison work?\")]\n",
    "chunks"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "6fd3e71b-439e-418f-8a8a-5232fba3d9fd",
   "metadata": {},
   "source": [
    "Stream just yielded the final result from that component.\n",
    "\n",
    "This is OK 🥹! Not all components have to implement streaming -- in some cases streaming is either unnecessary, difficult or just doesn't make sense.\n",
    "\n",
    ":::tip\n",
    "An LCEL chain constructed using non-streaming components, will still be able to stream in a lot of cases, with streaming of partial output starting after the last non-streaming step in the chain.\n",
    ":::"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 11,
   "id": "957447e6-1e60-41ef-8c10-2654bd9e738d",
   "metadata": {},
   "outputs": [],
   "source": [
    "retrieval_chain = (\n",
    "    {\n",
    "        \"context\": retriever.with_config(run_name=\"Docs\"),\n",
    "        \"question\": RunnablePassthrough(),\n",
    "    }\n",
    "    | prompt\n",
    "    | model\n",
    "    | StrOutputParser()\n",
    ")"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 12,
   "id": "94e50b5d-bf51-4eee-9da0-ee40dd9ce42b",
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Base|d on| the| given| context|,| Harrison| worke|d at| K|ens|ho|.|\n",
      "\n",
      "Here| are| |3| |made| up| sentences| about| this| place|:|\n",
      "\n",
      "1|.| K|ens|ho| was| a| cutting|-|edge| technology| company| known| for| its| innovative| solutions| in| artificial| intelligence| an|d data| analytics|.|\n",
      "\n",
      "2|.| The| modern| office| space| at| K|ens|ho| feature|d open| floor| plans|,| collaborative| work|sp|aces|,| an|d a| vib|rant| atmosphere| that| fos|tere|d creativity| an|d team|work|.|\n",
      "\n",
      "3|.| With| its| prime| location| in| the| heart| of| the| city|,| K|ens|ho| attracte|d top| talent| from| aroun|d the| worl|d,| creating| a| diverse| an|d dynamic| work| environment|.|"
     ]
    }
   ],
   "source": [
    "for chunk in retrieval_chain.stream(\n",
    "    \"Where did harrison work? \" \"Write 3 made up sentences about this place.\"\n",
    "):\n",
    "    print(chunk, end=\"|\", flush=True)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "8657aa4e-3469-4b5b-a09c-60b53a23b1e7",
   "metadata": {},
   "source": [
    "Now that we've seen how `stream` and `astream` work, let's venture into the world of streaming events. 🏞"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "baceb5c0-d4a4-4b98-8733-80ae4407b62d",
   "metadata": {},
   "source": [
    "## Using Stream Events\n",
    "\n",
    "Event Streaming is a **beta** API. This API may change a bit based on feedback.\n",
    "\n",
    ":::note\n",
    "\n",
    "This guide demonstrates the `V2` API and requires langchain-core >= 0.2. For the `V1` API compatible with older versions of LangChain, see [here](https://python.langchain.com/v0.1/docs/expression_language/streaming/#using-stream-events).\n",
    ":::"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "61348df9-ec58-401e-be89-68a70042f88e",
   "metadata": {},
   "outputs": [],
   "source": [
    "import langchain_core\n",
    "\n",
    "langchain_core.__version__"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "52e9e983-bbde-4906-9eca-4ccc06eabd91",
   "metadata": {},
   "source": [
    "For the `astream_events` API to work properly:\n",
    "\n",
    "* Use `async` throughout the code to the extent possible (e.g., async tools etc)\n",
    "* Propagate callbacks if defining custom functions / runnables\n",
    "* Whenever using runnables without LCEL, make sure to call `.astream()` on LLMs rather than `.ainvoke` to force the LLM to stream tokens.\n",
    "* Let us know if anything doesn't work as expected! :)\n",
    "\n",
    "### Event Reference\n",
    "\n",
    "Below is a reference table that shows some events that might be emitted by the various Runnable objects.\n",
    "\n",
    "\n",
    ":::note\n",
    "When streaming is implemented properly, the inputs to a runnable will not be known until after the input stream has been entirely consumed. This means that `inputs` will often be included only for `end` events and rather than for `start` events.\n",
    ":::\n",
    "\n",
    "| event                | name             | chunk                           | input                                         | output                                          |\n",
    "|----------------------|------------------|---------------------------------|-----------------------------------------------|-------------------------------------------------|\n",
    "| on_chat_model_start  | [model name]     |                                 | \\{\"messages\": [[SystemMessage, HumanMessage]]\\} |                                                 |\n",