{
 "cells": [
  {
   "cell_type": "markdown",
   "id": "13afcae7",
   "metadata": {},
   "source": [
    "# Redis\n",
    "\n",
    ">[Redis](https://redis.com) is an open-source key-value store that can be used as a cache, message broker, database, vector database and more.\n",
    "\n",
    "In the notebook, we'll demo the `SelfQueryRetriever` wrapped around a `Redis` vector store. "
   ]
  },
  {
   "cell_type": "markdown",
   "id": "68e75fb9",
   "metadata": {},
   "source": [
    "## Creating a Redis vector store\n",
    "First we'll want to create a Redis vector store and seed it with some data. We've created a small demo set of documents that contain summaries of movies.\n",
    "\n",
    "**Note:** The self-query retriever requires you to have `lark` installed (`pip install lark`) along with integration-specific requirements."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 1,
   "id": "63a8af5b",
   "metadata": {
    "tags": []
   },
   "outputs": [],
   "source": [
    "%pip install --upgrade --quiet  redis redisvl langchain-openai tiktoken lark"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "83811610-7df3-4ede-b268-68a6a83ba9e2",
   "metadata": {},
   "source": [
    "We want to use `OpenAIEmbeddings` so we have to get the OpenAI API Key."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 2,
   "id": "dd01b61b-7d32-4a55-85d6-b2d2d4f18840",
   "metadata": {
    "tags": []
   },
   "outputs": [],
   "source": [
    "import getpass\n",
    "import os\n",
    "\n",
    "if \"OPENAI_API_KEY\" not in os.environ:\n",
    "    os.environ[\"OPENAI_API_KEY\"] = getpass.getpass(\"OpenAI API Key:\")"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 3,
   "id": "cb4a5787",
   "metadata": {
    "tags": []
   },
   "outputs": [],
   "source": [
    "from langchain_community.vectorstores import Redis\n",
    "from langchain_core.documents import Document\n",
    "from langchain_openai import OpenAIEmbeddings\n",
    "\n",
    "embeddings = OpenAIEmbeddings()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 4,
   "id": "bcbe04d9",
   "metadata": {
    "tags": []
   },
   "outputs": [],
   "source": [
    "docs = [\n",
    "    Document(\n",
    "        page_content=\"A bunch of scientists bring back dinosaurs and mayhem breaks loose\",\n",
    "        metadata={\n",
    "            \"year\": 1993,\n",
    "            \"rating\": 7.7,\n",
    "            \"director\": \"Steven Spielberg\",\n",
    "            \"genre\": \"science fiction\",\n",
    "        },\n",
    "    ),\n",
    "    Document(\n",
    "        page_content=\"Leo DiCaprio gets lost in a dream within a dream within a dream within a ...\",\n",
    "        metadata={\n",
    "            \"year\": 2010,\n",
    "            \"director\": \"Christopher Nolan\",\n",
    "            \"genre\": \"science fiction\",\n",
    "            \"rating\": 8.2,\n",
    "        },\n",
    "    ),\n",
    "    Document(\n",
    "        page_content=\"A psychologist / detective gets lost in a series of dreams within dreams within dreams and Inception reused the idea\",\n",
    "        metadata={\n",
    "            \"year\": 2006,\n",
    "            \"director\": \"Satoshi Kon\",\n",
    "            \"genre\": \"science fiction\",\n",
    "            \"rating\": 8.6,\n",
    "        },\n",
    "    ),\n",
    "    Document(\n",
    "        page_content=\"A bunch of normal-sized women are supremely wholesome and some men pine after them\",\n",
    "        metadata={\n",
    "            \"year\": 2019,\n",
    "            \"director\": \"Greta Gerwig\",\n",
    "            \"genre\": \"drama\",\n",
    "            \"rating\": 8.3,\n",
    "        },\n",
    "    ),\n",
    "    Document(\n",
    "        page_content=\"Toys come alive and have a blast doing so\",\n",
    "        metadata={\n",
    "            \"year\": 1995,\n",
    "            \"director\": \"John Lasseter\",\n",
    "            \"genre\": \"animated\",\n",
    "            \"rating\": 9.1,\n",
    "        },\n",
    "    ),\n",
    "    Document(\n",
    "        page_content=\"Three men walk into the Zone, three men walk out of the Zone\",\n",
    "        metadata={\n",
    "            \"year\": 1979,\n",
    "            \"rating\": 9.9,\n",
    "            \"director\": \"Andrei Tarkovsky\",\n",
    "            \"genre\": \"science fiction\",\n",
    "        },\n",
    "    ),\n",
    "]"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 5,
   "id": "393aff3b",
   "metadata": {},
   "outputs": [
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "`index_schema` does not match generated metadata schema.\n",
      "If you meant to manually override the schema, please ignore this message.\n",
      "index_schema: {'tag': [{'name': 'genre'}], 'text': [{'name': 'director'}], 'numeric': [{'name': 'year'}, {'name': 'rating'}]}\n",
      "generated_schema: {'text': [{'name': 'director'}, {'name': 'genre'}], 'numeric': [{'name': 'year'}, {'name': 'rating'}], 'tag': []}\n",
      "\n"
     ]
    }
   ],
   "source": [
    "index_schema = {\n",
    "    \"tag\": [{\"name\": \"genre\"}],\n",
    "    \"text\": [{\"name\": \"director\"}],\n",
    "    \"numeric\": [{\"name\": \"year\"}, {\"name\": \"rating\"}],\n",
    "}\n",
    "\n",
    "vectorstore = Redis.from_documents(\n",
    "    docs,\n",
    "    embeddings,\n",
    "    redis_url=\"redis://localhost:6379\",\n",
    "    index_name=\"movie_reviews\",\n",
    "    index_schema=index_schema,\n",
    ")"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "5ecaab6d",
   "metadata": {},
   "source": [
    "## Creating our self-querying retriever\n",
    "Now we can instantiate our retriever. To do this we'll need to provide some information upfront about the metadata fields that our documents support and a short description of the document contents."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 6,
   "id": "86e34dbf",
   "metadata": {
    "tags": []
   },
   "outputs": [],
   "source": [
    "from langchain.chains.query_constructor.base import AttributeInfo\n",
    "from langchain.retrievers.self_query.base import SelfQueryRetriever\n",
    "from langchain_openai import OpenAI\n",
    "\n",
    "metadata_field_info = [\n",
    "    AttributeInfo(\n",
    "        name=\"genre\",\n",
    "        description=\"The genre of the movie\",\n",
    "        type=\"string or list[string]\",\n",
    "    ),\n",
    "    AttributeInfo(\n",
    "        name=\"year\",\n",
    "        description=\"The year the movie was released\",\n",
    "        type=\"integer\",\n",
    "    ),\n",
    "    AttributeInfo(\n",
    "        name=\"director\",\n",
    "        description=\"The name of the movie director\",\n",
    "        type=\"string\",\n",
    "    ),\n",
    "    AttributeInfo(\n",
    "        name=\"rating\", description=\"A 1-10 rating for the movie\", type=\"float\"\n",


{
 "cells": [
  {
   "cell_type": "markdown",
   "id": "dc23c48e",
   "metadata": {},
   "source": [
    "# Google Serper\n",
    "\n",
    "This notebook goes over how to use the `Google Serper` component to search the web. First you need to sign up for a free account at [serper.dev](https://serper.dev) and get your api key."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "ac0b9ce6",
   "metadata": {},
   "outputs": [],
   "source": [
    "%pip install --upgrade --quiet  langchain-community"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 11,
   "id": "a8acfb24",
   "metadata": {
    "ExecuteTime": {
     "end_time": "2023-05-04T00:56:29.336521Z",
     "start_time": "2023-05-04T00:56:29.334173Z"
    },
    "collapsed": false,
    "jupyter": {
     "outputs_hidden": false
    },
    "pycharm": {
     "is_executing": true
    }
   },
   "outputs": [],
   "source": [
    "import os\n",
    "import pprint\n",
    "\n",
    "os.environ[\"SERPER_API_KEY\"] = \"\""
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 2,
   "id": "54bf5afd",
   "metadata": {
    "ExecuteTime": {
     "end_time": "2023-05-04T00:54:07.676293Z",
     "start_time": "2023-05-04T00:54:06.665742Z"
    }
   },
   "outputs": [],
   "source": [
    "from langchain_community.utilities import GoogleSerperAPIWrapper"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 3,
   "id": "31f8f382",
   "metadata": {
    "ExecuteTime": {
     "end_time": "2023-05-04T00:54:08.324245Z",
     "start_time": "2023-05-04T00:54:08.321577Z"
    }
   },
   "outputs": [],
   "source": [
    "search = GoogleSerperAPIWrapper()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 4,
   "id": "25ce0225",
   "metadata": {
    "ExecuteTime": {
     "end_time": "2023-05-04T00:54:11.399847Z",
     "start_time": "2023-05-04T00:54:09.335597Z"
    }
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "'Barack Hussein Obama II'"
      ]
     },
     "execution_count": 4,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "search.run(\"Obama's first name?\")"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "1f1c6c22",
   "metadata": {},
   "source": [
    "## As part of a Self Ask With Search Chain"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 5,
   "id": "c1b5edd7",
   "metadata": {
    "ExecuteTime": {
     "end_time": "2023-05-04T00:54:14.311773Z",
     "start_time": "2023-05-04T00:54:14.304389Z"
    },
    "collapsed": false,
    "jupyter": {
     "outputs_hidden": false
    }
   },
   "outputs": [],
   "source": [
    "os.environ[\"OPENAI_API_KEY\"] = \"\""
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 5,
   "id": "a8ccea61",
   "metadata": {
    "collapsed": false,
    "jupyter": {
     "outputs_hidden": false
    }
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "\n",
      "\n",
      "\u001b[1m> Entering new AgentExecutor chain...\u001b[0m\n",
      "\u001b[32;1m\u001b[1;3m Yes.\n",
      "Follow up: Who is the reigning men's U.S. Open champion?\u001b[0m\n",
      "Intermediate answer: \u001b[36;1m\u001b[1;3mCurrent champions Carlos Alcaraz, 2022 men's singles champion.\u001b[0m\n",
      "\u001b[32;1m\u001b[1;3mFollow up: Where is Carlos Alcaraz from?\u001b[0m\n",
      "Intermediate answer: \u001b[36;1m\u001b[1;3mEl Palmar, Spain\u001b[0m\n",
      "\u001b[32;1m\u001b[1;3mSo the final answer is: El Palmar, Spain\u001b[0m\n",
      "\n",
      "\u001b[1m> Finished chain.\u001b[0m\n"
     ]
    },
    {
     "data": {
      "text/plain": [
       "'El Palmar, Spain'"
      ]
     },
     "execution_count": 5,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "from langchain.agents import AgentType, initialize_agent\n",
    "from langchain_community.utilities import GoogleSerperAPIWrapper\n",
    "from langchain_core.tools import Tool\n",
    "from langchain_openai import OpenAI\n",
    "\n",
    "llm = OpenAI(temperature=0)\n",
    "search = GoogleSerperAPIWrapper()\n",
    "tools = [\n",
    "    Tool(\n",
    "        name=\"Intermediate Answer\",\n",
    "        func=search.run,\n",
    "        description=\"useful for when you need to ask with search\",\n",
    "    )\n",
    "]\n",
    "\n",
    "self_ask_with_search = initialize_agent(\n",
    "    tools, llm, agent=AgentType.SELF_ASK_WITH_SEARCH, verbose=True\n",
    ")\n",
    "self_ask_with_search.run(\n",
    "    \"What is the hometown of the reigning men's U.S. Open champion?\"\n",
    ")"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "3aee3682",
   "metadata": {},
   "source": [
    "## Obtaining results with metadata\n",
    "If you would also like to obtain the results in a structured way including metadata. For this we will be using the `results` method of the wrapper."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 6,
   "id": "073c3fc5",
   "metadata": {
    "ExecuteTime": {
     "end_time": "2023-05-04T00:54:22.863413Z",
     "start_time": "2023-05-04T00:54:20.827395Z"
    },
    "collapsed": false,
    "jupyter": {
     "outputs_hidden": false
    },
    "pycharm": {
     "is_executing": true
    }
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "{'searchParameters': {'q': 'Apple Inc.',\n",
      "                      'gl': 'us',\n",
      "                      'hl': 'en',\n",
      "                      'num': 10,\n",
      "                      'type': 'search'},\n",
      " 'knowledgeGraph': {'title': 'Apple',\n",
      "                    'type': 'Technology company',\n",
      "                    'website': 'http://www.apple.com/',\n",
      "                    'imageUrl': 'https://encrypted-tbn0.gstatic.com/images?q=tbn:ANd9GcQwGQRv5TjjkycpctY66mOg_e2-npacrmjAb6_jAWhzlzkFE3OTjxyzbA&s=0',\n",
      "                    'description': 'Apple Inc. is an American multinational '\n",
      "                                   'technology company headquartered in '\n",
      "                                   'Cupertino, California. Apple is the '\n",
      "                                   \"world's largest technology company by \"\n",


{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "4x4kQ0VcodAC"
   },
   "source": [
    "# Exa Search"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "V1x8wEUhodAH"
   },
   "source": [
    "Exa is a search engine fully designed for use by LLMs. Search for documents on the internet using **natural language queries**, then retrieve **cleaned HTML content** from desired documents.\n",
    "\n",
    "Unlike keyword-based search (Google), Exa's neural search capabilities allow it to semantically understand queries and return relevant documents. For example, we could search `\"fascinating article about cats\"` and compare the search results from [Google](https://www.google.com/search?q=fascinating+article+about+cats) and [Exa](https://search.exa.ai/search?q=fascinating%20article%20about%20cats&autopromptString=Here%20is%20a%20fascinating%20article%20about%20cats%3A). Google gives us SEO-optimized listicles based on the keyword \"fascinating\". Exa just works.\n",
    "\n",
    "This notebook goes over how to use Exa Search with LangChain.\n",
    "\n",
    "First, get an Exa API key and add it as an environment variable. Get $10 free credit (plus more by completing certain actions like making your first search) by [signing up here](https://dashboard.exa.ai/)."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 1,
   "metadata": {
    "id": "7GON_jdvodAI"
   },
   "outputs": [],
   "source": [
    "import os\n",
    "\n",
    "api_key = os.getenv(\"EXA_API_KEY\")  # Set your API key as an environment variable"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "And install the integration package"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "%pip install --upgrade --quiet langchain-exa \n",
    "\n",
    "# and some deps for this notebook\n",
    "%pip install --upgrade --quiet langchain langchain-openai langchain-community"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Using ExaSearchRetriever\n",
    "\n",
    "ExaSearchRetriever is a retriever that uses Exa Search to retrieve relevant documents."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    ":::note\n",
    "\n",
    "The `max_characters` parameter for **TextContentsOptions** used to be called `max_length` which is now deprecated. Make sure to use `max_characters` instead.\n",
    "\n",
    ":::"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "ip5_D9MkodAK"
   },
   "source": [
    "## Using the Exa SDK as LangChain Agent Tools\n",
    "\n",
    "The [Exa SDK](https://docs.exa.ai/) creates a client that can interact with three main Exa API endpoints:\n",
    "\n",
    "- `search`: Given a natural language search query, retrieve a list of search results.\n",
    "- `find_similar`: Given a URL, retrieve a list of search results corresponding to webpages which are similar to the document at the provided URL.\n",
    "- `get_contents`: Given a list of document ids fetched from `search` or `find_similar`, get cleaned HTML content for each document.\n",
    "\n",
    "The `exa_py` SDK combines these endpoints into two powerful calls. Using these provide the most flexible and efficient use cases of Exa search:\n",
    "\n",
    "1. `search_and_contents`: Combines the `search` and `get_contents` endpoints to retrieve search results along with their content in a single operation.\n",
    "2. `find_similar_and_contents`: Combines the `find_similar` and `get_contents` endpoints to find similar pages and retrieve their content in one call.\n",
    "\n",
    "We can use the `@tool` decorator and docstrings to create LangChain Tool wrappers that tell an LLM agent how to use these combined Exa functionalities effectively. This approach simplifies usage and reduces the number of API calls needed to get comprehensive results.\n",
    "\n",
    "Before writing code, ensure you have `langchain-exa` installed"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "id": "0CsJFtuFodAK"
   },
   "outputs": [],
   "source": [
    "%pip install --upgrade --quiet  langchain-exa"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 5,
   "metadata": {
    "id": "XatXTApdodAM"
   },
   "outputs": [],
   "source": [
    "import os\n",
    "\n",
    "from exa_py import Exa\n",
    "from langchain_core.tools import tool\n",
    "\n",
    "exa = Exa(api_key=os.environ[\"EXA_API_KEY\"])\n",
    "\n",
    "\n",
    "@tool\n",
    "def search_and_contents(query: str):\n",
    "    \"\"\"Search for webpages based on the query and retrieve their contents.\"\"\"\n",
    "    # This combines two API endpoints: search and contents retrieval\n",
    "    return exa.search_and_contents(\n",
    "        query, use_autoprompt=True, num_results=5, text=True, highlights=True\n",
    "    )\n",
    "\n",
    "\n",
    "@tool\n",
    "def find_similar_and_contents(url: str):\n",
    "    \"\"\"Search for webpages similar to a given URL and retrieve their contents.\n",
    "    The url passed in should be a URL returned from `search_and_contents`.\n",
    "    \"\"\"\n",
    "    # This combines two API endpoints: find similar and contents retrieval\n",
    "    return exa.find_similar_and_contents(url, num_results=5, text=True, highlights=True)\n",
    "\n",
    "\n",
    "tools = [search_and_contents, find_similar_and_contents]"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "sVe2ca9OodAO"
   },
   "source": [
    "### Providing Exa Tools to an Agent\n",
    "\n",
    "We can provide the Exa tools we just created to a LangChain `OpenAIFunctionsAgent`. When asked to `Summarize for me a fascinating article about cats`, the agent uses the `search` tool to perform a Exa search with an appropriate search query, uses the `get_contents` tool to perform Exa content retrieval, and then returns a summary of the retrieved content."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "id": "x72pY95WodAP"
   },
   "outputs": [],
   "source": [
    "from langchain.agents import AgentExecutor, OpenAIFunctionsAgent\n",
    "from langchain_core.messages import SystemMessage\n",
    "from langchain_openai import ChatOpenAI\n",
    "\n",
    "llm = ChatOpenAI(temperature=0)\n",
    "\n",
    "system_message = SystemMessage(\n",
    "    content=\"You are a web researcher who answers user questions by looking up information on the internet and retrieving contents of helpful documents. Cite your sources.\"\n",
    ")\n",
    "\n",
    "agent_prompt = OpenAIFunctionsAgent.create_prompt(system_message)\n",
    "agent = OpenAIFunctionsAgent(llm=llm, tools=tools, prompt=agent_prompt)\n",
    "agent_executor = AgentExecutor(agent=agent, tools=tools, verbose=True)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 16,
   "metadata": {
    "colab": {
     "base_uri": "https://localhost:8080/",
     "height": 1000
    },
    "id": "jTku-3OmodAP",
    "outputId": "d587ed66-3b82-4bc8-e8cd-f5278e0c14a6",
    "scrolled": false
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "\n",


{
 "cells": [
  {
   "cell_type": "markdown",
   "id": "0e499e90-7a6d-4fab-8aab-31a4df417601",
   "metadata": {},
   "source": [
    "# SQLDatabase Toolkit\n",
    "\n",
    "This will help you getting started with the SQL Database [toolkit](/docs/concepts/#toolkits). For detailed documentation of all `SQLDatabaseToolkit` features and configurations head to the [API reference](https://python.langchain.com/api_reference/community/agent_toolkits/langchain_community.agent_toolkits.sql.toolkit.SQLDatabaseToolkit.html).\n",
    "\n",
    "Tools within the `SQLDatabaseToolkit` are designed to interact with a `SQL` database. \n",
    "\n",
    "A common application is to enable agents to answer questions using data in a relational database, potentially in an iterative fashion (e.g., recovering from errors).\n",
    "\n",
    "**⚠ Security note ⚠**\n",
    "\n",
    "Building Q&A systems of SQL databases requires executing model-generated SQL queries. There are inherent risks in doing this. Make sure that your database connection permissions are always scoped as narrowly as possible for your chain/agent's needs. This will mitigate though not eliminate the risks of building a model-driven system. For more on general security best practices, [see here](/docs/security).\n",
    "\n",
    "## Setup\n",
    "\n",
    "If you want to get automated tracing from runs of individual tools, you can also set your [LangSmith](https://docs.smith.langchain.com/) API key by uncommenting below:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "3de6e3be-1fd9-42a3-8564-8ca7dca11e1c",
   "metadata": {},
   "outputs": [],
   "source": [
    "# os.environ[\"LANGSMITH_API_KEY\"] = getpass.getpass(\"Enter your LangSmith API key: \")\n",
    "# os.environ[\"LANGSMITH_TRACING\"] = \"true\""
   ]
  },
  {
   "cell_type": "markdown",
   "id": "31896b61-68d2-4b4d-be9d-b829eda327d1",
   "metadata": {},
   "source": [
    "### Installation\n",
    "\n",
    "This toolkit lives in the `langchain-community` package:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "c4933e04-9120-4ccc-9ef7-369987823b0e",
   "metadata": {},
   "outputs": [],
   "source": [
    "%pip install --upgrade --quiet  langchain-community"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "6ad08dbe-1642-448c-b58d-153810024375",
   "metadata": {},
   "source": [
    "For demonstration purposes, we will access a prompt in the LangChain [Hub](https://smith.langchain.com/hub). We will also require `langgraph` to demonstrate the use of the toolkit with an agent. This is not required to use the toolkit."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "f3dead45-9908-497d-a5a3-bce30642e88f",
   "metadata": {},
   "outputs": [],
   "source": [
    "%pip install --upgrade --quiet langchainhub langgraph"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "804533b1-2f16-497b-821b-c82d67fcf7b6",
   "metadata": {},
   "source": [
    "## Instantiation\n",
    "\n",
    "The `SQLDatabaseToolkit` toolkit requires:\n",
    "\n",
    "- a [SQLDatabase](https://python.langchain.com/api_reference/community/utilities/langchain_community.utilities.sql_database.SQLDatabase.html) object;\n",
    "- a LLM or chat model (for instantiating the [QuerySQLCheckerTool](https://python.langchain.com/api_reference/community/tools/langchain_community.tools.sql_database.tool.QuerySQLCheckerTool.html) tool).\n",
    "\n",
    "Below, we instantiate the toolkit with these objects. Let's first create a database object.\n",
    "\n",
    "This guide uses the example `Chinook` database based on [these instructions](https://database.guide/2-sample-databases-sqlite/).\n",
    "\n",
    "Below we will use the `requests` library to pull the `.sql` file and create an in-memory SQLite database. Note that this approach is lightweight, but ephemeral and not thread-safe. If you'd prefer, you can follow the instructions to save the file locally as `Chinook.db` and instantiate the database via `db = SQLDatabase.from_uri(\"sqlite:///Chinook.db\")`."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 1,
   "id": "40d05f9b-5a8f-4307-8f8b-4153db0fdfa9",
   "metadata": {},
   "outputs": [],
   "source": [
    "import sqlite3\n",
    "\n",
    "import requests\n",
    "from langchain_community.utilities.sql_database import SQLDatabase\n",
    "from sqlalchemy import create_engine\n",
    "from sqlalchemy.pool import StaticPool\n",
    "\n",
    "\n",
    "def get_engine_for_chinook_db():\n",
    "    \"\"\"Pull sql file, populate in-memory database, and create engine.\"\"\"\n",
    "    url = \"https://raw.githubusercontent.com/lerocha/chinook-database/master/ChinookDatabase/DataSources/Chinook_Sqlite.sql\"\n",
    "    response = requests.get(url)\n",
    "    sql_script = response.text\n",
    "\n",
    "    connection = sqlite3.connect(\":memory:\", check_same_thread=False)\n",
    "    connection.executescript(sql_script)\n",
    "    return create_engine(\n",
    "        \"sqlite://\",\n",
    "        creator=lambda: connection,\n",
    "        poolclass=StaticPool,\n",
    "        connect_args={\"check_same_thread\": False},\n",
    "    )\n",
    "\n",
    "\n",
    "engine = get_engine_for_chinook_db()\n",
    "\n",
    "db = SQLDatabase(engine)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "2b9a6326-78fd-4c42-a1cb-4316619ac449",
   "metadata": {},
   "source": [
    "We will also need a LLM or chat model:\n",
    "\n",
    "import ChatModelTabs from \"@theme/ChatModelTabs\";\n",
    "\n",
    "<ChatModelTabs customVarName=\"llm\" />\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 2,
   "id": "cc6e6108-83d9-404f-8f31-474c2fbf5f6c",
   "metadata": {},
   "outputs": [],
   "source": [
    "# | output: false\n",
    "# | echo: false\n",
    "\n",
    "from langchain_openai import ChatOpenAI\n",
    "\n",
    "llm = ChatOpenAI(temperature=0)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "77925e72-4730-43c3-8726-d68cedf635f4",
   "metadata": {},
   "source": [
    "We can now instantiate the toolkit:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 3,
   "id": "42bd5a41-672a-4a53-b70a-2f0c0555758c",
   "metadata": {},
   "outputs": [],
   "source": [
    "from langchain_community.agent_toolkits.sql.toolkit import SQLDatabaseToolkit\n",
    "\n",
    "toolkit = SQLDatabaseToolkit(db=db, llm=llm)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "b2f882cf-4156-4a9f-a714-db97ec8ccc37",
   "metadata": {},
   "source": [
    "## Tools\n",
    "\n",
    "View available tools:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 4,
   "id": "a18c3e69-bee0-4f5d-813e-eeb540f41b98",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [


"execution_count": 11,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "import getpass\n",
    "import os\n",
    "\n",
    "from langchain import hub\n",
    "from langchain.agents import AgentExecutor, create_tool_calling_agent\n",
    "from langchain_openai import AzureChatOpenAI\n",
    "\n",
    "os.environ[\"AZURE_OPENAI_API_KEY\"] = getpass.getpass()\n",
    "os.environ[\"AZURE_OPENAI_ENDPOINT\"] = \"https://<your-endpoint>.openai.azure.com/\"\n",
    "os.environ[\"AZURE_OPENAI_API_VERSION\"] = \"2023-06-01-preview\"\n",
    "os.environ[\"AZURE_OPENAI_DEPLOYMENT_NAME\"] = \"<your-deployment-name>\"\n",
    "\n",
    "instructions = \"\"\"You are an assistant.\"\"\"\n",
    "base_prompt = hub.pull(\"langchain-ai/openai-functions-template\")\n",
    "prompt = base_prompt.partial(instructions=instructions)\n",
    "llm = AzureChatOpenAI(\n",
    "    openai_api_key=os.environ[\"AZURE_OPENAI_API_KEY\"],\n",
    "    azure_endpoint=os.environ[\"AZURE_OPENAI_ENDPOINT\"],\n",
    "    azure_deployment=os.environ[\"AZURE_OPENAI_DEPLOYMENT_NAME\"],\n",
    "    openai_api_version=os.environ[\"AZURE_OPENAI_API_VERSION\"],\n",
    ")\n",
    "tool = BingSearchResults(api_wrapper=api_wrapper)\n",
    "tools = [tool]\n",
    "agent = create_tool_calling_agent(llm, tools, prompt)\n",
    "agent_executor = AgentExecutor(\n",
    "    agent=agent,\n",
    "    tools=tools,\n",
    "    verbose=True,\n",
    ")\n",
    "agent_executor.invoke({\"input\": \"What happened in the latest burning man floods?\"})"
   ]
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 3 (ipykernel)",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.10.14"
  },
  "vscode": {
   "interpreter": {
    "hash": "a0a0263b650d907a3bfe41c0f8d6a63a071b884df3cfdc1579f00cdc1aed6b03"
   }
  }
 },
 "nbformat": 4,
 "nbformat_minor": 4
}


"metadata": {
    "tags": []
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "'https://web.archive.org/web/20230428133211/https://cnn.com/world'"
      ]
     },
     "execution_count": 8,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# If the agent wants to remember the current webpage, it can use the `current_webpage` tool\n",
    "await tools_by_name[\"current_webpage\"].arun({})"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Use within an Agent\n",
    "\n",
    "Several of the browser tools are `StructuredTool`'s, meaning they expect multiple arguments. These aren't compatible (out of the box) with agents older than the `STRUCTURED_CHAT_ZERO_SHOT_REACT_DESCRIPTION`"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 10,
   "metadata": {
    "tags": []
   },
   "outputs": [
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "/Users/isaachershenson/.pyenv/versions/3.11.9/lib/python3.11/site-packages/langchain_core/_api/deprecation.py:139: LangChainDeprecationWarning: The function `initialize_agent` was deprecated in LangChain 0.1.0 and will be removed in 0.3.0. Use Use new agent constructor methods like create_react_agent, create_json_agent, create_structured_chat_agent, etc. instead.\n",
      "  warn_deprecated(\n"
     ]
    }
   ],
   "source": [
    "from langchain.agents import AgentType, initialize_agent\n",
    "from langchain_anthropic import ChatAnthropic\n",
    "\n",
    "llm = ChatAnthropic(\n",
    "    model_name=\"claude-3-haiku-20240307\", temperature=0\n",
    ")  # or any other LLM, e.g., ChatOpenAI(), OpenAI()\n",
    "\n",
    "agent_chain = initialize_agent(\n",
    "    tools,\n",
    "    llm,\n",
    "    agent=AgentType.STRUCTURED_CHAT_ZERO_SHOT_REACT_DESCRIPTION,\n",
    "    verbose=True,\n",
    ")"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 11,
   "metadata": {
    "tags": []
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "\n",
      "\n",
      "\u001b[1m> Entering new AgentExecutor chain...\u001b[0m\n",
      "\u001b[32;1m\u001b[1;3mThought: To find the headers on langchain.com, I will navigate to the website and extract the text.\n",
      "\n",
      "Action:\n",
      "```\n",
      "{\n",
      "  \"action\": \"navigate_browser\",\n",
      "  \"action_input\": \"https://langchain.com\"\n",
      "}\n",
      "```\n",
      "\n",
      "\u001b[0m\n",
      "Observation: \u001b[33;1m\u001b[1;3mNavigating to https://langchain.com returned status code 200\u001b[0m\n",
      "Thought:\u001b[32;1m\u001b[1;3mOkay, let's find the headers on the langchain.com website.\n",
      "\n",
      "Action:\n",
      "```\n",
      "{\n",
      "  \"action\": \"extract_text\",\n",
      "  \"action_input\": {}\n",
      "}\n",
      "```\n",
      "\n",
      "\u001b[0m\n",
      "Observation: \u001b[31;1m\u001b[1;3mLangChain We value your privacy We use cookies to analyze our traffic. By clicking \"Accept All\", you consent to our use of cookies. Privacy Policy Customize Reject All Accept All Customize Consent Preferences We may use cookies to help you navigate efficiently and perform certain functions. You will find detailed information about all cookies under each consent category below. The cookies that are categorized as \"Necessary\" are stored on your browser as they are essential for enabling the basic functionalities of the site.... Show more Necessary Always Active Necessary cookies are required to enable the basic features of this site, such as providing secure log-in or adjusting your consent preferences. These cookies do not store any personally identifiable data. Functional Functional cookies help perform certain functionalities like sharing the content of the website on social media platforms, collecting feedback, and other third-party features. Analytics Analytical cookies are used to understand how visitors interact with the website. These cookies help provide information on metrics such as the number of visitors, bounce rate, traffic source, etc. Performance Performance cookies are used to understand and analyze the key performance indexes of the website which helps in delivering a better user experience for the visitors. Advertisement Advertisement cookies are used to provide visitors with customized advertisements based on the pages you visited previously and to analyze the effectiveness of the ad campaigns. Uncategorized Other uncategorized cookies are those that are being analyzed and have not been classified into a category as yet. Reject All Save My Preferences Accept All Products LangChain LangSmith LangGraph Methods Retrieval Agents Evaluation Resources Blog Case Studies Use Case Inspiration Experts Changelog Docs LangChain Docs LangSmith Docs Company About Careers Pricing Get a demo Sign up LangChain’s suite of products supports developers along each step of the LLM application lifecycle. Applications that can reason. Powered by LangChain. Get a demo Sign up for free From startups to global enterprises, ambitious builders choose LangChain products. Build LangChain is a framework to build with LLMs by chaining interoperable components. LangGraph is the framework for building controllable agentic workflows. Run Deploy your LLM applications at scale with LangGraph Cloud, our infrastructure purpose-built for agents. Manage Debug, collaborate, test, and monitor your LLM app in LangSmith - whether it's built with a LangChain framework or not. Build your app with LangChain Build context-aware, reasoning applications with LangChain’s flexible framework that leverages your company’s data and APIs. Future-proof your application by making vendor optionality part of your LLM infrastructure design. Learn more about LangChain Run at scale with LangGraph Cloud Deploy your LangGraph app with LangGraph Cloud for fault-tolerant scalability - including support for async background jobs, built-in persistence, and distributed task queues. Learn more about LangGraph Manage LLM performance with LangSmith Ship faster with LangSmith’s debug, test, deploy, and monitoring workflows. Don’t rely on “vibes” – add engineering rigor to your LLM-development workflow, whether you’re building with LangChain or not. Learn more about LangSmith Hear from our happy customers LangChain, LangGraph, and LangSmith help teams of all sizes, across all industries - from ambitious startups to established enterprises. “LangSmith helped us improve the accuracy and performance of Retool’s fine-tuned models. Not only did we deliver a better product by iterating with LangSmith, but we’re shipping new AI features to our users in a fraction of the time it would have taken without it.” Jamie Cuffe Head of Self-Serve and New Products “By combining the benefits of LangSmith and standing on the shoulders of a gigantic open-source community, we’re able to identify the right approaches of using LLMs in an enterprise-setting faster.” Yusuke Kaji General Manager of AI “Working with LangChain and LangSmith on the Elastic AI Assistant had a significant positive impact on the overall pace and quality of the development and shipping experience. We couldn’t have achieved  the product experience delivered to our customers without LangChain, and we couldn’t have done it at the same pace without LangSmith.” James Spiteri Director of Security Products “As soon as we heard about LangSmith, we moved our entire development stack onto it. We could have built evaluation, testing and monitoring tools in house, but with LangSmith it took us 10x less time to get a 1000x better tool.” Jose Peña Senior Manager The reference architecture enterprises adopt for success. LangChain’s suite of products can be used independently or stacked together for multiplicative impact – guiding you through building, running, and managing your LLM apps. 15M+ Monthly Downloads 100K+ Apps Powered 75K+ GitHub Stars 3K+ Contributors The biggest developer community in GenAI Learn alongside the 1M+ developers who are pushing the industry forward. Explore LangChain Get started with the LangSmith platform today Get a demo Sign up for free Teams building with LangChain are driving operational efficiency, increasing discovery & personalization, and delivering premium products that generate revenue. Discover Use Cases Get inspired by companies who have done it. Financial Services FinTech Technology LangSmith is the enterprise DevOps platform built for LLMs. Explore LangSmith Gain visibility to make trade offs between cost, latency, and quality. Increase developer productivity. Eliminate manual, error-prone testing. Reduce hallucinations and improve reliability. Enterprise deployment options to keep data secure. Ready to start shipping
",


{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Human as a tool\n",
    "\n",
    "Human are AGI so they can certainly be used as a tool to help out AI agent \n",
    "when it is confused."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "%pip install --upgrade --quiet  langchain-community"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 1,
   "metadata": {
    "tags": []
   },
   "outputs": [],
   "source": [
    "from langchain.agents import AgentType, initialize_agent, load_tools\n",
    "from langchain_openai import ChatOpenAI, OpenAI\n",
    "\n",
    "llm = ChatOpenAI(temperature=0.0)\n",
    "math_llm = OpenAI(temperature=0.0)\n",
    "tools = load_tools(\n",
    "    [\"human\", \"llm-math\"],\n",
    "    llm=math_llm,\n",
    ")\n",
    "\n",
    "agent_chain = initialize_agent(\n",
    "    tools,\n",
    "    llm,\n",
    "    agent=AgentType.ZERO_SHOT_REACT_DESCRIPTION,\n",
    "    verbose=True,\n",
    ")"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "In the above code you can see the tool takes input directly from command line.\n",
    "You can customize `prompt_func` and `input_func` according to your need (as shown below)."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 2,
   "metadata": {
    "tags": []
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "\n",
      "\n",
      "\u001b[1m> Entering new AgentExecutor chain...\u001b[0m\n",
      "\u001b[32;1m\u001b[1;3mI don't know Eric's surname, so I should ask a human for guidance.\n",
      "Action: Human\n",
      "Action Input: \"What is Eric's surname?\"\u001b[0m\n",
      "\n",
      "What is Eric's surname?\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      " Zhu\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "\n",
      "Observation: \u001b[36;1m\u001b[1;3mZhu\u001b[0m\n",
      "Thought:\u001b[32;1m\u001b[1;3mI now know Eric's surname is Zhu.\n",
      "Final Answer: Eric's surname is Zhu.\u001b[0m\n",
      "\n",
      "\u001b[1m> Finished chain.\u001b[0m\n"
     ]
    },
    {
     "data": {
      "text/plain": [
       "\"Eric's surname is Zhu.\""
      ]
     },
     "execution_count": 2,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "agent_chain.run(\"What's my friend Eric's surname?\")\n",
    "# Answer with 'Zhu'"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Configuring the Input Function\n",
    "\n",
    "By default, the `HumanInputRun` tool uses the python `input` function to get input from the user.\n",
    "You can customize the input_func to be anything you'd like.\n",
    "For instance, if you want to accept multi-line input, you could do the following:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 8,
   "metadata": {
    "tags": []
   },
   "outputs": [],
   "source": [
    "def get_input() -> str:\n",
    "    print(\"Insert your text. Enter 'q' or press Ctrl-D (or Ctrl-Z on Windows) to end.\")\n",
    "    contents = []\n",
    "    while True:\n",
    "        try:\n",
    "            line = input()\n",
    "        except EOFError:\n",
    "            break\n",
    "        if line == \"q\":\n",
    "            break\n",
    "        contents.append(line)\n",
    "    return \"\\n\".join(contents)\n",
    "\n",
    "\n",
    "# You can modify the tool when loading\n",
    "tools = load_tools([\"human\", \"ddg-search\"], llm=math_llm, input_func=get_input)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 9,
   "metadata": {
    "tags": []
   },
   "outputs": [],
   "source": [
    "# Or you can directly instantiate the tool\n",
    "from langchain_community.tools import HumanInputRun\n",
    "\n",
    "tool = HumanInputRun(input_func=get_input)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 10,
   "metadata": {
    "tags": []
   },
   "outputs": [],
   "source": [
    "agent_chain = initialize_agent(\n",
    "    tools,\n",
    "    llm,\n",
    "    agent=AgentType.ZERO_SHOT_REACT_DESCRIPTION,\n",
    "    verbose=True,\n",
    ")"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 12,
   "metadata": {
    "tags": []
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "\n",
      "\n",
      "\u001b[1m> Entering new AgentExecutor chain...\u001b[0m\n",
      "\u001b[32;1m\u001b[1;3mI should ask a human for guidance\n",
      "Action: Human\n",
      "Action Input: \"Can you help me attribute a quote?\"\u001b[0m\n",
      "\n",
      "Can you help me attribute a quote?\n",
      "Insert your text. Enter 'q' or press Ctrl-D (or Ctrl-Z on Windows) to end.\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      " vini\n",
      " vidi\n",
      " vici\n",
      " q\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "\n",
      "Observation: \u001b[36;1m\u001b[1;3mvini\n",
      "vidi\n",
      "vici\u001b[0m\n",
      "Thought:\u001b[32;1m\u001b[1;3mI need to provide more context about the quote\n",
      "Action: Human\n",
      "Action Input: \"The quote is 'Veni, vidi, vici'\"\u001b[0m\n",
      "\n",
      "The quote is 'Veni, vidi, vici'\n",
      "Insert your text. Enter 'q' or press Ctrl-D (or Ctrl-Z on Windows) to end.\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      " oh who said it \n",
      " q\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "\n",
      "Observation: \u001b[36;1m\u001b[1;3moh who said it \u001b[0m\n",
      "Thought:\u001b[32;1m\u001b[1;3mI can use DuckDuckGo Search to find out who said the quote\n",
      "Action: DuckDuckGo Search\n",
      "Action Input: \"Who said 'Veni, vidi, vici'?\"\u001b[0m\n",


{
 "cells": [
  {
   "cell_type": "markdown",
   "id": "c81da886",
   "metadata": {},
   "source": [
    "# Pandas Dataframe\n",
    "\n",
    "This notebook shows how to use agents to interact with a `Pandas DataFrame`. It is mostly optimized for question answering.\n",
    "\n",
    "**NOTE: this agent calls the `Python` agent under the hood, which executes LLM generated Python code - this can be bad if the LLM generated Python code is harmful. Use cautiously.**"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 1,
   "id": "0cdd9bf5",
   "metadata": {},
   "outputs": [],
   "source": [
    "from langchain.agents.agent_types import AgentType\n",
    "from langchain_experimental.agents.agent_toolkits import create_pandas_dataframe_agent\n",
    "from langchain_openai import ChatOpenAI"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 2,
   "id": "051ebe84",
   "metadata": {},
   "outputs": [],
   "source": [
    "import pandas as pd\n",
    "from langchain_openai import OpenAI\n",
    "\n",
    "df = pd.read_csv(\n",
    "    \"https://raw.githubusercontent.com/pandas-dev/pandas/main/doc/data/titanic.csv\"\n",
    ")"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "a62858e2",
   "metadata": {},
   "source": [
    "## Using `ZERO_SHOT_REACT_DESCRIPTION`\n",
    "\n",
    "This shows how to initialize the agent using the `ZERO_SHOT_REACT_DESCRIPTION` agent type."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 3,
   "id": "4185ff46",
   "metadata": {},
   "outputs": [],
   "source": [
    "agent = create_pandas_dataframe_agent(OpenAI(temperature=0), df, verbose=True)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "7233ab56",
   "metadata": {},
   "source": [
    "## Using OpenAI Functions\n",
    "\n",
    "This shows how to initialize the agent using the OPENAI_FUNCTIONS agent type. Note that this is an alternative to the above."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 4,
   "id": "a8ea710e",
   "metadata": {},
   "outputs": [],
   "source": [
    "agent = create_pandas_dataframe_agent(\n",
    "    ChatOpenAI(temperature=0, model=\"gpt-3.5-turbo-0613\"),\n",
    "    df,\n",
    "    verbose=True,\n",
    "    agent_type=AgentType.OPENAI_FUNCTIONS,\n",
    ")"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 5,
   "id": "a9207a2e",
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "\n",
      "\n",
      "\u001b[1m> Entering new  chain...\u001b[0m\n",
      "\u001b[32;1m\u001b[1;3m\n",
      "Invoking: `python_repl_ast` with `df.shape[0]`\n",
      "\n",
      "\n",
      "\u001b[0m\u001b[36;1m\u001b[1;3m891\u001b[0m\u001b[32;1m\u001b[1;3mThere are 891 rows in the dataframe.\u001b[0m\n",
      "\n",
      "\u001b[1m> Finished chain.\u001b[0m\n"
     ]
    },
    {
     "data": {
      "text/plain": [
       "'There are 891 rows in the dataframe.'"
      ]
     },
     "execution_count": 5,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "agent.invoke(\"how many rows are there?\")"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 5,
   "id": "bd43617c",
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "\n",
      "\n",
      "\u001b[1m> Entering new AgentExecutor chain...\u001b[0m\n",
      "\u001b[32;1m\u001b[1;3mThought: I need to count the number of people with more than 3 siblings\n",
      "Action: python_repl_ast\n",
      "Action Input: df[df['SibSp'] > 3].shape[0]\u001b[0m\n",
      "Observation: \u001b[36;1m\u001b[1;3m30\u001b[0m\n",
      "Thought:\u001b[32;1m\u001b[1;3m I now know the final answer\n",
      "Final Answer: 30 people have more than 3 siblings.\u001b[0m\n",
      "\n",
      "\u001b[1m> Finished chain.\u001b[0m\n"
     ]
    },
    {
     "data": {
      "text/plain": [
       "'30 people have more than 3 siblings.'"
      ]
     },
     "execution_count": 5,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "agent.invoke(\"how many people have more than 3 siblings\")"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 6,
   "id": "94e64b58",
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "\n",
      "\n",
      "\u001b[1m> Entering new AgentExecutor chain...\u001b[0m\n",
      "\u001b[32;1m\u001b[1;3mThought: I need to calculate the average age first\n",
      "Action: python_repl_ast\n",
      "Action Input: df['Age'].mean()\u001b[0m\n",
      "Observation: \u001b[36;1m\u001b[1;3m29.69911764705882\u001b[0m\n",
      "Thought:\u001b[32;1m\u001b[1;3m I now need to calculate the square root of the average age\n",
      "Action: python_repl_ast\n",
      "Action Input: math.sqrt(df['Age'].mean())\u001b[0m\n",
      "Observation: \u001b[36;1m\u001b[1;3mNameError(\"name 'math' is not defined\")\u001b[0m\n",
      "Thought:\u001b[32;1m\u001b[1;3m I need to import the math library\n",
      "Action: python_repl_ast\n",
      "Action Input: import math\u001b[0m\n",
      "Observation: \u001b[36;1m\u001b[1;3m\u001b[0m\n",
      "Thought:\u001b[32;1m\u001b[1;3m I now need to calculate the square root of the average age\n",
      "Action: python_repl_ast\n",
      "Action Input: math.sqrt(df['Age'].mean())\u001b[0m\n",
      "Observation: \u001b[36;1m\u001b[1;3m5.449689683556195\u001b[0m\n",
      "Thought:\u001b[32;1m\u001b[1;3m I now know the final answer\n",
      "Final Answer: The square root of the average age is 5.449689683556195.\u001b[0m\n",
      "\n",
      "\u001b[1m> Finished chain.\u001b[0m\n"
     ]
    },
    {
     "data": {
      "text/plain": [
       "'The square root of the average age is 5.449689683556195.'"
      ]
     },
     "execution_count": 6,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "agent.invoke(\"whats the square root of the average age?\")"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "c4bc0584",
   "metadata": {},
   "source": [


"File \u001b[0;32m~/job/integrations/langchain/libs/partners/openai/langchain_openai/chat_models/__init__.py:1\u001b[0m\n\u001b[0;32m----> 1\u001b[0m \u001b[38;5;28;01mfrom\u001b[39;00m \u001b[38;5;21;01mlangchain_openai\u001b[39;00m\u001b[38;5;21;01m.\u001b[39;00m\u001b[38;5;21;01mchat_models\u001b[39;00m\u001b[38;5;21;01m.\u001b[39;00m\u001b[38;5;21;01mazure\u001b[39;00m \u001b[38;5;28;01mimport\u001b[39;00m AzureChatOpenAI\n\u001b[1;32m      2\u001b[0m \u001b[38;5;28;01mfrom\u001b[39;00m \u001b[38;5;21;01mlangchain_openai\u001b[39;00m\u001b[38;5;21;01m.\u001b[39;00m\u001b[38;5;21;01mchat_models\u001b[39;00m\u001b[38;5;21;01m.\u001b[39;00m\u001b[38;5;21;01mbase\u001b[39;00m \u001b[38;5;28;01mimport\u001b[39;00m ChatOpenAI\n\u001b[1;32m      4\u001b[0m __all__ \u001b[38;5;241m=\u001b[39m [\n\u001b[1;32m      5\u001b[0m     \u001b[38;5;124m\"\u001b[39m\u001b[38;5;124mChatOpenAI\u001b[39m\u001b[38;5;124m\"\u001b[39m,\n\u001b[1;32m      6\u001b[0m     \u001b[38;5;124m\"\u001b[39m\u001b[38;5;124mAzureChatOpenAI\u001b[39m\u001b[38;5;124m\"\u001b[39m,\n\u001b[1;32m      7\u001b[0m ]\n",
      "File \u001b[0;32m~/job/integrations/langchain/libs/partners/openai/langchain_openai/chat_models/azure.py:8\u001b[0m\n\u001b[1;32m      5\u001b[0m \u001b[38;5;28;01mimport\u001b[39;00m \u001b[38;5;21;01mos\u001b[39;00m\n\u001b[1;32m      6\u001b[0m \u001b[38;5;28;01mfrom\u001b[39;00m \u001b[38;5;21;01mtyping\u001b[39;00m \u001b[38;5;28;01mimport\u001b[39;00m Any, Callable, Dict, List, Optional, Union\n\u001b[0;32m----> 8\u001b[0m \u001b[38;5;28;01mimport\u001b[39;00m \u001b[38;5;21;01mopenai\u001b[39;00m\n\u001b[1;32m      9\u001b[0m \u001b[38;5;28;01mfrom\u001b[39;00m \u001b[38;5;21;01mlangchain_core\u001b[39;00m\u001b[38;5;21;01m.\u001b[39;00m\u001b[38;5;21;01moutputs\u001b[39;00m \u001b[38;5;28;01mimport\u001b[39;00m ChatResult\n\u001b[1;32m     10\u001b[0m \u001b[38;5;28;01mfrom\u001b[39;00m \u001b[38;5;21;01mlangchain_core\u001b[39;00m\u001b[38;5;21;01m.\u001b[39;00m\u001b[38;5;21;01mpydantic_v1\u001b[39;00m \u001b[38;5;28;01mimport\u001b[39;00m Field, SecretStr, root_validator\n",
      "File \u001b[0;32m~/job/zep-proprietary/venv/lib/python3.11/site-packages/openai/__init__.py:8\u001b[0m\n\u001b[1;32m      5\u001b[0m \u001b[38;5;28;01mimport\u001b[39;00m \u001b[38;5;21;01mos\u001b[39;00m \u001b[38;5;28;01mas\u001b[39;00m \u001b[38;5;21;01m_os\u001b[39;00m\n\u001b[1;32m      6\u001b[0m \u001b[38;5;28;01mfrom\u001b[39;00m \u001b[38;5;21;01mtyping_extensions\u001b[39;00m \u001b[38;5;28;01mimport\u001b[39;00m override\n\u001b[0;32m----> 8\u001b[0m \u001b[38;5;28;01mfrom\u001b[39;00m \u001b[38;5;21;01m.\u001b[39;00m \u001b[38;5;28;01mimport\u001b[39;00m types\n\u001b[1;32m      9\u001b[0m \u001b[38;5;28;01mfrom\u001b[39;00m \u001b[38;5;21;01m.\u001b[39;00m\u001b[38;5;21;01m_types\u001b[39;00m \u001b[38;5;28;01mimport\u001b[39;00m NOT_GIVEN, NoneType, NotGiven, Transport, ProxiesTypes\n\u001b[1;32m     10\u001b[0m \u001b[38;5;28;01mfrom\u001b[39;00m \u001b[38;5;21;01m.\u001b[39;00m\u001b[38;5;21;01m_utils\u001b[39;00m \u001b[38;5;28;01mimport\u001b[39;00m file_from_path\n",


>[Streamlit](https://docs.streamlit.io/) is an open-source Python library that makes it easy to create and share beautiful,

custom web apps for machine learning and data science.

This notebook goes over how to store and use chat message history in a `Streamlit` app. `StreamlitChatMessageHistory` will store messages in

at the specified `key=`. The default key is `"langchain_messages"`.

- Note, `StreamlitChatMessageHistory` only works when run in a Streamlit app.

- You may also be interested in [StreamlitCallbackHandler](/docs/integrations/callbacks/streamlit) for LangChain.

The integration lives in the `langchain-community` package, so we need to install that. We also need to install `streamlit`.

pip install -U langchain-community streamlit

You can see the [full app example running here](https://langchain-st-memory.streamlit.app/), and more examples in

from langchain_community.chat_message_histories import (

We can easily combine this message history class with [LCEL Runnables](/docs/how_to/message_history).

The history will be persisted across re-runs of the Streamlit app within a given user session. A given `StreamlitChatMessageHistory` will NOT be persisted or shared across user sessions.

# Optionally, specify your own session_state key for storing messages

msgs.add_ai_message("How can I help you?")

from langchain_core.prompts import ChatPromptTemplate, MessagesPlaceholder

from langchain_core.runnables.history import RunnableWithMessageHistory

("system", "You are an AI chatbot having a conversation with a human."),

lambda session_id: msgs,  # Always return the instance created earlier

Conversational Streamlit apps will often re-draw each previous chat message on every re-run. This is easy to do by iterating through `StreamlitChatMessageHistory.messages`:

# As usual, new messages are added to StreamlitChatMessageHistory when the Chain is called.

config = {"configurable": {"session_id": "any"}}

response = chain_with_history.invoke({"question": prompt}, config)

**[View the final app](https://langchain-st-memory.streamlit.app/).**


"We can [chain](/docs/how_to/sequence/) our model with a prompt template like so:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 5,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "AIMessage(content='Ich liebe das Programmieren.', response_metadata={'id': 'item0', 'partial': False, 'value': {'completion': 'Ich liebe das Programmieren.', 'logprobs': {'text_offset': [], 'top_logprobs': []}, 'prompt': '<|start_header_id|>system<|end_header_id|>\\n\\nYou are a helpful assistant that translates English to German.<|eot_id|><|start_header_id|>user<|end_header_id|>\\n\\nI love programming.<|eot_id|><|start_header_id|>assistant<|end_header_id|>\\n\\n', 'stop_reason': 'end_of_text', 'tokens': ['Ich', ' liebe', ' das', ' Programm', 'ieren', '.'], 'total_tokens_count': 36}, 'params': {}, 'status': None}, id='item0')"
      ]
     },
     "execution_count": 5,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "from langchain_core.prompts import ChatPromptTemplate\n",
    "\n",
    "prompt = ChatPromptTemplate(\n",
    "    [\n",
    "        (\n",
    "            \"system\",\n",
    "            \"You are a helpful assistant that translates {input_language} to {output_language}.\",\n",
    "        ),\n",
    "        (\"human\", \"{input}\"),\n",
    "    ]\n",
    ")\n",
    "\n",
    "chain = prompt | llm\n",
    "chain.invoke(\n",
    "    {\n",
    "        \"input_language\": \"English\",\n",
    "        \"output_language\": \"German\",\n",
    "        \"input\": \"I love programming.\",\n",
    "    }\n",
    ")"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Streaming"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 6,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Arrr, ye landlubber! Ye be wantin' to learn about owls, eh? Well, matey, settle yerself down with a pint o' grog and listen close, for I be tellin' ye about these fascinatin' creatures o' the night!\n",
      "\n",
      "Owls be birds, but not just any birds, me hearty! They be nocturnal, meanin' they do their huntin' at night, when the rest o' the world be sleepin'. And they be experts at it, too! Their big, round eyes be designed for seein' in the dark, with a special reflective layer called the tapetum lucidum that helps 'em spot prey in the shadows. It's like havin' a built-in lantern, savvy?\n",
      "\n",
      "But that be not all, me matey! Owls also have acute hearin', which helps 'em pinpoint the slightest sounds in the dark. And their ears be asymmetrical, meanin' one ear be higher than the other, which gives 'em better depth perception. It's like havin' a built-in sonar system, arrr!\n",
      "\n",
      "Now, ye might be wonderin' how owls fly so silently, like ghosts in the night. Well, it be because o' their special feathers, me hearty! They have soft, fringed feathers on their wings that help reduce noise and turbulence, makin' 'em the sneakiest flyers on the seven seas... er, skies!\n",
      "\n",
      "Owls come in all shapes and sizes, from the tiny elf owl to the great grey owl, which be one o' the largest owl species in the world. And they be found on every continent, except Antarctica, o' course. They be solitary creatures, but some species be known to form long-term monogamous relationships, like the barn owl and its mate.\n",
      "\n",
      "So, there ye have it, me hearty! Owls be amazin' creatures, with their clever adaptations and stealthy ways. Now, go forth and spread the word about these magnificent birds o' the night! And remember, if ye ever encounter an owl in the wild, be sure to show respect and keep a weather eye open, or ye might just find yerself on the receivin' end o' a silent, flyin' tackle! Arrr!"
     ]
    }
   ],
   "source": [
    "system = \"You are a helpful assistant with pirate accent.\"\n",
    "human = \"I want to learn more about this animal: {animal}\"\n",
    "prompt = ChatPromptTemplate.from_messages([(\"system\", system), (\"human\", human)])\n",
    "\n",
    "chain = prompt | llm\n",
    "\n",
    "for chunk in chain.stream({\"animal\": \"owl\"}):\n",
    "    print(chunk.content, end=\"\", flush=True)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Async"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 7,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "AIMessage(content='The capital of France is Paris.', response_metadata={'id': 'item0', 'partial': False, 'value': {'completion': 'The capital of France is Paris.', 'logprobs': {'text_offset': [], 'top_logprobs': []}, 'prompt': '<|start_header_id|>user<|end_header_id|>\\n\\nwhat is the capital of France?<|eot_id|><|start_header_id|>assistant<|end_header_id|>\\n\\n', 'stop_reason': 'end_of_text', 'tokens': ['The', ' capital', ' of', ' France', ' is', ' Paris', '.'], 'total_tokens_count': 24}, 'params': {}, 'status': None}, id='item0')"
      ]
     },
     "execution_count": 7,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "prompt = ChatPromptTemplate.from_messages(\n",
    "    [\n",
    "        (\n",
    "            \"human\",\n",
    "            \"what is the capital of {country}?\",\n",
    "        )\n",
    "    ]\n",
    ")\n",
    "\n",
    "chain = prompt | llm\n",
    "await chain.ainvoke({\"country\": \"France\"})"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Async Streaming"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 8,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Quantum computers use quantum bits (qubits) to process multiple possibilities simultaneously, exponentially faster than classical computers, enabling breakthroughs in fields like cryptography, optimization, and simulation."
     ]
    }
   ],
   "source": [
    "prompt = ChatPromptTemplate.from_messages(\n",
    "    [\n",
    "        (\n",
    "            \"human\",\n",
    "            \"in less than {num_words} words explain me {topic} \",\n",
    "        )\n",
    "    ]\n",
    ")\n",
    "chain = prompt | llm\n",
    "\n",
    "async for chunk in chain.astream({\"num_words\": 30, \"topic\": \"quantum computers\"}):\n",
    "    print(chunk.content, end=\"\", flush=True)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## API reference\n",
    "\n",
    "For detailed documentation of all ChatSambaStudio features and configurations head to the API reference: https://api.python.langchain.com/en/latest/chat_models/langchain_community.chat_models.sambanova.ChatSambaStudio.html"
   ]
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "langchain",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",


This notebook shows how to use LangChain with [LlamaAPI](https://llama-api.com/) - a hosted version of Llama2 that adds in support for function calling.

%pip install --upgrade --quiet  llamaapi

# Replace 'Your_API_Token' with your actual API token

stderr

/Users/harrisonchase/.pyenv/versions/3.9.1/envs/langchain/lib/python3.9/site-packages/deeplake/util/check_latest_version.py:32: UserWarning: A newer version of deeplake (3.6.12) is available. It's recommended that you update to the latest version using `pip install -U deeplake`.

warnings.warn(

from langchain_experimental.llms import ChatLlamaAPI

from langchain.chains import create_tagging_chain

"description": "the sentiment encountered in the passage",

"description": "a 0-10 score of how aggressive the passage is",

"language": {"type": "string", "description": "the language of the passage"},

chain = create_tagging_chain(schema, model)

{'sentiment': 'aggressive', 'aggressiveness': 8, 'language': 'english'}


{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# ChatHuggingFace\n",
    "\n",
    "This will help you getting started with `langchain_huggingface` [chat models](/docs/concepts/#chat-models). For detailed documentation of all `ChatHuggingFace` features and configurations head to the [API reference](https://python.langchain.com/api_reference/huggingface/chat_models/langchain_huggingface.chat_models.huggingface.ChatHuggingFace.html). For a list of models supported by Hugging Face check out [this page](https://huggingface.co/models).\n",
    "\n",
    "## Overview\n",
    "### Integration details\n",
    "\n",
    "### Integration details\n",
    "\n",
    "| Class | Package | Local | Serializable | JS support | Package downloads | Package latest |\n",
    "| :--- | :--- | :---: | :---: |  :---: | :---: | :---: |\n",
    "| [ChatHuggingFace](https://python.langchain.com/api_reference/huggingface/chat_models/langchain_huggingface.chat_models.huggingface.ChatHuggingFace.html) | [langchain-huggingface](https://python.langchain.com/api_reference/huggingface/index.html) | ✅ | beta | ❌ | ![PyPI - Downloads](https://img.shields.io/pypi/dm/langchain_huggingface?style=flat-square&label=%20) | ![PyPI - Version](https://img.shields.io/pypi/v/langchain_huggingface?style=flat-square&label=%20) |\n",
    "\n",
    "### Model features\n",
    "| [Tool calling](/docs/how_to/tool_calling) | [Structured output](/docs/how_to/structured_output/) | JSON mode | [Image input](/docs/how_to/multimodal_inputs/) | Audio input | Video input | [Token-level streaming](/docs/how_to/chat_streaming/) | Native async | [Token usage](/docs/how_to/chat_token_usage_tracking/) | [Logprobs](/docs/how_to/logprobs/) |\n",
    "| :---: | :---: | :---: | :---: |  :---: | :---: | :---: | :---: | :---: | :---: |\n",
    "| ✅ | ✅ | ❌ | ✅ | ✅ | ✅ | ❌ | ✅ | ✅ | ❌ | \n",
    "\n",
    "## Setup\n",
    "\n",
    "To access Hugging Face models you'll need to create a Hugging Face account, get an API key, and install the `langchain-huggingface` integration package.\n",
    "\n",
    "### Credentials\n",
    "\n",
    "Generate a [Hugging Face Access Token](https://huggingface.co/docs/hub/security-tokens) and store it as an environment variable: `HUGGINGFACEHUB_API_TOKEN`."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "import getpass\n",
    "import os\n",
    "\n",
    "if not os.getenv(\"HUGGINGFACEHUB_API_TOKEN\"):\n",
    "    os.environ[\"HUGGINGFACEHUB_API_TOKEN\"] = getpass.getpass(\"Enter your token: \")"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Installation\n",
    "\n",
    "| Class | Package | Local | Serializable | JS support | Package downloads | Package latest |\n",
    "| :--- | :--- | :---: | :---: |  :---: | :---: | :---: |\n",
    "| [ChatHuggingFace](https://python.langchain.com/api_reference/huggingface/chat_models/langchain_huggingface.chat_models.huggingface.ChatHuggingFace.html) | [langchain_huggingface](https://python.langchain.com/api_reference/huggingface/index.html) | ✅ | ❌ | ❌ | ![PyPI - Downloads](https://img.shields.io/pypi/dm/langchain_huggingface?style=flat-square&label=%20) | ![PyPI - Version](https://img.shields.io/pypi/v/langchain_huggingface?style=flat-square&label=%20) |\n",
    "\n",
    "### Model features\n",
    "| [Tool calling](/docs/how_to/tool_calling) | [Structured output](/docs/how_to/structured_output/) | JSON mode | [Image input](/docs/how_to/multimodal_inputs/) | Audio input | Video input | [Token-level streaming](/docs/how_to/chat_streaming/) | Native async | [Token usage](/docs/how_to/chat_token_usage_tracking/) | [Logprobs](/docs/how_to/logprobs/) |\n",
    "| :---: | :---: | :---: | :---: |  :---: | :---: | :---: | :---: | :---: | :---: |\n",
    "| ✅ | ✅ | ❌ | ❌ | ❌ | ❌ | ❌ | ❌ | ❌ | ❌ | \n",
    "\n",
    "## Setup\n",
    "\n",
    "To access `langchain_huggingface` models you'll need to create a/an `Hugging Face` account, get an API key, and install the `langchain_huggingface` integration package.\n",
    "\n",
    "### Credentials\n",
    "\n",
    "You'll need to have a [Hugging Face Access Token](https://huggingface.co/docs/hub/security-tokens) saved as an environment variable: `HUGGINGFACEHUB_API_TOKEN`."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 1,
   "metadata": {},
   "outputs": [],
   "source": [
    "import getpass\n",
    "import os\n",
    "\n",
    "os.environ[\"HUGGINGFACEHUB_API_TOKEN\"] = getpass.getpass(\n",
    "    \"Enter your Hugging Face API key: \"\n",
    ")"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 5,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "\n",
      "\u001b[1m[\u001b[0m\u001b[34;49mnotice\u001b[0m\u001b[1;39;49m]\u001b[0m\u001b[39;49m A new release of pip is available: \u001b[0m\u001b[31;49m24.0\u001b[0m\u001b[39;49m -> \u001b[0m\u001b[32;49m24.1.2\u001b[0m\n",
      "\u001b[1m[\u001b[0m\u001b[34;49mnotice\u001b[0m\u001b[1;39;49m]\u001b[0m\u001b[39;49m To update, run: \u001b[0m\u001b[32;49mpip install --upgrade pip\u001b[0m\n",
      "Note: you may need to restart the kernel to use updated packages.\n"
     ]
    }
   ],
   "source": [
    "%pip install --upgrade --quiet  langchain-huggingface text-generation transformers google-search-results numexpr langchainhub sentencepiece jinja2 bitsandbytes accelerate"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Instantiation\n",
    "\n",
    "You can instantiate a `ChatHuggingFace` model in two different ways, either from a `HuggingFaceEndpoint` or from a `HuggingFacePipeline`."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### `HuggingFaceEndpoint`"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 10,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "The token has not been saved to the git credentials helper. Pass `add_to_git_credential=True` in this function directly or `--add-to-git-credential` if using via `huggingface-cli` if you want to set the git credential as well.\n",
      "Token is valid (permission: fineGrained).\n",
      "Your token has been saved to /Users/isaachershenson/.cache/huggingface/token\n",
      "Login successful\n"
     ]
    }
   ],
   "source": [
    "from langchain_huggingface import ChatHuggingFace, HuggingFaceEndpoint\n",
    "\n",
    "llm = HuggingFaceEndpoint(\n",


"    load_in_4bit=True,\n",
    "    bnb_4bit_quant_type=\"nf4\",\n",
    "    bnb_4bit_compute_dtype=\"float16\",\n",
    "    bnb_4bit_use_double_quant=True,\n",
    ")"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "and pass it to the `HuggingFacePipeline` as a part of its `model_kwargs`:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "llm = HuggingFacePipeline.from_model_id(\n",
    "    model_id=\"HuggingFaceH4/zephyr-7b-beta\",\n",
    "    task=\"text-generation\",\n",
    "    pipeline_kwargs=dict(\n",
    "        max_new_tokens=512,\n",
    "        do_sample=False,\n",
    "        repetition_penalty=1.03,\n",
    "        return_full_text=False,\n",
    "    ),\n",
    "    model_kwargs={\"quantization_config\": quantization_config},\n",
    ")\n",
    "\n",
    "chat_model = ChatHuggingFace(llm=llm)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Invocation"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 11,
   "metadata": {},
   "outputs": [],
   "source": [
    "from langchain_core.messages import (\n",
    "    HumanMessage,\n",
    "    SystemMessage,\n",
    ")\n",
    "\n",
    "messages = [\n",
    "    SystemMessage(content=\"You're a helpful assistant\"),\n",
    "    HumanMessage(\n",
    "        content=\"What happens when an unstoppable force meets an immovable object?\"\n",
    "    ),\n",
    "]\n",
    "\n",
    "ai_msg = chat_model.invoke(messages)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 12,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "According to the popular phrase and hypothetical scenario, when an unstoppable force meets an immovable object, a paradoxical situation arises as both forces are seemingly contradictory. On one hand, an unstoppable force is an entity that cannot be stopped or prevented from moving forward, while on the other hand, an immovable object is something that cannot be moved or displaced from its position. \n",
      "\n",
      "In this scenario, it is un\n"
     ]
    }
   ],
   "source": [
    "print(ai_msg.content)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## API reference\n",
    "\n",
    "For detailed documentation of all `ChatHuggingFace` features and configurations head to the API reference: https://python.langchain.com/api_reference/huggingface/chat_models/langchain_huggingface.chat_models.huggingface.ChatHuggingFace.html"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## API reference\n",
    "\n",
    "For detailed documentation of all ChatHuggingFace features and configurations head to the API reference: https://python.langchain.com/api_reference/huggingface/chat_models/langchain_huggingface.chat_models.huggingface.ChatHuggingFace.html"
   ]
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 3 (ipykernel)",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.11.9"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 4
}


{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Llama.cpp\n",
    "\n",
    ">[llama.cpp python](https://github.com/abetlen/llama-cpp-python) library is a simple Python bindings for `@ggerganov`\n",
    ">[llama.cpp](https://github.com/ggerganov/llama.cpp).\n",
    ">\n",
    ">This package provides:\n",
    ">\n",
    "> - Low-level access to C API via ctypes interface.\n",
    "> - High-level Python API for text completion\n",
    ">   - `OpenAI`-like API\n",
    ">   - `LangChain` compatibility\n",
    ">   - `LlamaIndex` compatibility\n",
    "> - OpenAI compatible web server\n",
    ">   - Local Copilot replacement\n",
    ">   - Function Calling support\n",
    ">   - Vision API support\n",
    ">   - Multiple Models\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Overview\n",
    "\n",
    "### Integration details\n",
    "| Class | Package | Local | Serializable | JS support |\n",
    "| :--- | :--- | :---: | :---: |  :---: |\n",
    "| [ChatLlamaCpp](https://python.langchain.com/api_reference/community/chat_models/langchain_community.chat_models.llamacpp.ChatLlamaCpp.html) | [langchain-community](https://python.langchain.com/api_reference/community/index.html) | ✅ | ❌ | ❌ |\n",
    "\n",
    "### Model features\n",
    "| [Tool calling](/docs/how_to/tool_calling) | [Structured output](/docs/how_to/structured_output/) | JSON mode | Image input | Audio input | Video input | [Token-level streaming](/docs/how_to/chat_streaming/) | Native async | [Token usage](/docs/how_to/chat_token_usage_tracking/) | [Logprobs](/docs/how_to/logprobs/) |\n",
    "| :---: | :---: | :---: | :---: |  :---: | :---: | :---: | :---: | :---: | :---: |\n",
    "| ✅ | ✅ | ❌ | ❌ | ❌ | ❌ | ✅ | ❌ | ❌ | ✅ | \n",
    "\n",
    "## Setup\n",
    "\n",
    "To get started and use **all** the features show below, we reccomend using a model that has been fine-tuned for tool-calling.\n",
    "\n",
    "We will use [\n",
    "Hermes-2-Pro-Llama-3-8B-GGUF](https://huggingface.co/NousResearch/Hermes-2-Pro-Llama-3-8B-GGUF) from NousResearch. \n",
    "\n",
    "> Hermes 2 Pro is an upgraded version of Nous Hermes 2, consisting of an updated and cleaned version of the OpenHermes 2.5 Dataset, as well as a newly introduced Function Calling and JSON Mode dataset developed in-house. This new version of Hermes maintains its excellent general task and conversation capabilities - but also excels at Function Calling\n",
    "\n",
    "See our guides on local models to go deeper:\n",
    "\n",
    "* [Run LLMs locally](https://python.langchain.com/v0.1/docs/guides/development/local_llms/)\n",
    "* [Using local models with RAG](https://python.langchain.com/v0.1/docs/use_cases/question_answering/local_retrieval_qa/)\n",
    "\n",
    "### Installation\n",
    "\n",
    "The LangChain LlamaCpp integration lives in the `langchain-community` and `llama-cpp-python` packages:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "%pip install -qU langchain-community llama-cpp-python"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Instantiation\n",
    "\n",
    "Now we can instantiate our model object and generate chat completions:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 14,
   "metadata": {},
   "outputs": [],
   "source": [
    "# Path to your model weights\n",
    "local_model = \"local/path/to/Hermes-2-Pro-Llama-3-8B-Q8_0.gguf\""
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "import multiprocessing\n",
    "\n",
    "from langchain_community.chat_models import ChatLlamaCpp\n",
    "\n",
    "llm = ChatLlamaCpp(\n",
    "    temperature=0.5,\n",
    "    model_path=local_model,\n",
    "    n_ctx=10000,\n",
    "    n_gpu_layers=8,\n",
    "    n_batch=300,  # Should be between 1 and n_ctx, consider the amount of VRAM in your GPU.\n",
    "    max_tokens=512,\n",
    "    n_threads=multiprocessing.cpu_count() - 1,\n",
    "    repeat_penalty=1.5,\n",
    "    top_p=0.5,\n",
    "    verbose=True,\n",
    ")"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Invocation"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "messages = [\n",
    "    (\n",
    "        \"system\",\n",
    "        \"You are a helpful assistant that translates English to French. Translate the user sentence.\",\n",
    "    ),\n",
    "    (\"human\", \"I love programming.\"),\n",
    "]\n",
    "\n",
    "ai_msg = llm.invoke(messages)\n",
    "ai_msg"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 4,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "J'aime programmer. (In France, \"programming\" is often used in its original sense of scheduling or organizing events.) \n",
      "\n",
      "If you meant computer-programming: \n",
      "Je suis amoureux de la programmation informatique.\n",
      "\n",
      "(You might also say simply 'programmation', which would be understood as both meanings - depending on context).\n"
     ]
    }
   ],
   "source": [
    "print(ai_msg.content)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Chaining\n",
    "\n",
    "We can [chain](/docs/how_to/sequence/) our model with a prompt template like so:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "from langchain_core.prompts import ChatPromptTemplate\n",
    "\n",
    "prompt = ChatPromptTemplate.from_messages(\n",
    "    [\n",
    "        (\n",
    "            \"system\",\n",
    "            \"You are a helpful assistant that translates {input_language} to {output_language}.\",\n",
    "        ),\n",
    "        (\"human\", \"{input}\"),\n",
    "    ]\n",
    ")\n",
    "\n",
    "chain = prompt | llm\n",
    "chain.invoke(\n",
    "    {\n",
    "        \"input_language\": \"English\",\n",
    "        \"output_language\": \"German\",\n",
    "        \"input\": \"I love programming.\",\n",
    "    }\n",
    ")"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Tool calling\n",
    "\n",
    "Firstly, it works mostly the same as OpenAI Function Calling\n",
    "\n",
    "OpenAI has a [tool calling](https://platform.openai.com/docs/guides/function-calling) (we use \"tool calling\" and \"function calling\" interchangeably here) API that lets you describe tools and their arguments, and have the model return a JSON object with a tool to invoke and the inputs to that tool. tool-calling is extremely useful for building tool-using chains and agents, and for getting structured outputs from models more generally.\n",
    "\n",


This notebook covers how to get started with JinaChat chat models.

from langchain_community.chat_models import JinaChat

from langchain_core.messages import HumanMessage, SystemMessage

from langchain_core.prompts.chat import (

AIMessage(content="J'aime programmer.", additional_kwargs={}, example=False)

content="You are a helpful assistant that translates English to French."

content="Translate this sentence from English to French. I love programming."

You can make use of templating by using a `MessagePromptTemplate`. You can build a `ChatPromptTemplate` from one or more `MessagePromptTemplates`. You can use `ChatPromptTemplate`'s `format_prompt` -- this returns a `PromptValue`, which you can convert to a string or Message object, depending on whether you want to use the formatted value as input to an llm or chat model.

For convenience, there is a `from_template` method exposed on the template. If you were to use this template, this is what it would look like:

"You are a helpful assistant that translates {input_language} to {output_language}."

AIMessage(content="J'aime programmer.", additional_kwargs={}, example=False)

# get a chat completion from the formatted messages

input_language="English", output_language="French", text="I love programming."


{
 "cells": [
  {
   "cell_type": "raw",
   "id": "eb65deaa",
   "metadata": {},
   "source": [
    "---\n",
    "sidebar_label: vLLM Chat\n",
    "---"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "8f82e243-f4ee-44e2-b417-099b6401ae3e",
   "metadata": {},
   "source": [
    "# vLLM Chat\n",
    "\n",
    "vLLM can be deployed as a server that mimics the OpenAI API protocol. This allows vLLM to be used as a drop-in replacement for applications using OpenAI API. This server can be queried in the same format as OpenAI API.\n",
    "\n",
    "## Overview\n",
    "This will help you getting started with vLLM [chat models](/docs/concepts/#chat-models), which leverage the `langchain-openai` package. For detailed documentation of all `ChatOpenAI` features and configurations head to the [API reference](https://python.langchain.com/api_reference/openai/chat_models/langchain_openai.chat_models.base.ChatOpenAI.html).\n",
    "\n",
    "### Integration details\n",
    "\n",
    "| Class | Package | Local | Serializable | JS support | Package downloads | Package latest |\n",
    "| :--- | :--- | :---: | :---: |  :---: | :---: | :---: |\n",
    "| [ChatOpenAI](https://python.langchain.com/api_reference/openai/chat_models/langchain_openai.chat_models.base.ChatOpenAI.html) | [langchain_openai](https://python.langchain.com/api_reference/openai/) | ✅ | beta | ❌ | ![PyPI - Downloads](https://img.shields.io/pypi/dm/langchain_openai?style=flat-square&label=%20) | ![PyPI - Version](https://img.shields.io/pypi/v/langchain_openai?style=flat-square&label=%20) |\n",
    "\n",
    "### Model features\n",
    "Specific model features-- such as tool calling, support for multi-modal inputs, support for token-level streaming, etc.-- will depend on the hosted model.\n",
    "\n",
    "## Setup\n",
    "\n",
    "See the vLLM docs [here](https://docs.vllm.ai/en/latest/).\n",
    "\n",
    "To access vLLM models through LangChain, you'll need to install the `langchain-openai` integration package.\n",
    "\n",
    "### Credentials\n",
    "\n",
    "Authentication will depend on specifics of the inference server."
   ]
  },
  {
   "cell_type": "markdown",
   "id": "c3b1707a-cf2c-4367-94e3-436c43402503",
   "metadata": {},
   "source": [
    "If you want to get automated tracing of your model calls you can also set your [LangSmith](https://docs.smith.langchain.com/) API key by uncommenting below:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "1e40bd5e-cbaa-41ef-aaf9-0858eb207184",
   "metadata": {},
   "outputs": [],
   "source": [
    "# os.environ[\"LANGCHAIN_TRACING_V2\"] = \"true\"\n",
    "# os.environ[\"LANGCHAIN_API_KEY\"] = getpass.getpass(\"Enter your LangSmith API key: \")"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "0739b647-609b-46d3-bdd3-e86fe4463288",
   "metadata": {},
   "source": [
    "### Installation\n",
    "\n",
    "The LangChain vLLM integration can be accessed via the `langchain-openai` package:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "7afcfbdc-56aa-4529-825a-8acbe7aa5241",
   "metadata": {},
   "outputs": [],
   "source": [
    "%pip install -qU langchain-openai"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "2cf576d6-7b67-4937-bf99-39071e85720c",
   "metadata": {},
   "source": [
    "## Instantiation\n",
    "\n",
    "Now we can instantiate our model object and generate chat completions:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 1,
   "id": "060a2e3d-d42f-4221-bd09-a9a06544dcd3",
   "metadata": {
    "tags": []
   },
   "outputs": [],
   "source": [
    "from langchain_core.messages import HumanMessage, SystemMessage\n",
    "from langchain_core.prompts.chat import (\n",
    "    ChatPromptTemplate,\n",
    "    HumanMessagePromptTemplate,\n",
    "    SystemMessagePromptTemplate,\n",
    ")\n",
    "from langchain_openai import ChatOpenAI"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 14,
   "id": "bf24d732-68a9-44fd-b05d-4903ce5620c6",
   "metadata": {
    "tags": []
   },
   "outputs": [],
   "source": [
    "inference_server_url = \"http://localhost:8000/v1\"\n",
    "\n",
    "llm = ChatOpenAI(\n",
    "    model=\"mosaicml/mpt-7b\",\n",
    "    openai_api_key=\"EMPTY\",\n",
    "    openai_api_base=inference_server_url,\n",
    "    max_tokens=5,\n",
    "    temperature=0,\n",
    ")"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "34b18328-5e8b-4ff2-9b89-6fbb76b5c7f0",
   "metadata": {},
   "source": [
    "## Invocation"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 15,
   "id": "aea4e363-5688-4b07-82ed-6aa8153c2377",
   "metadata": {
    "tags": []
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "AIMessage(content=' Io amo programmare', additional_kwargs={}, example=False)"
      ]
     },
     "execution_count": 15,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "messages = [\n",
    "    SystemMessage(\n",
    "        content=\"You are a helpful assistant that translates English to Italian.\"\n",
    "    ),\n",
    "    HumanMessage(\n",
    "        content=\"Translate the following sentence from English to Italian: I love programming.\"\n",
    "    ),\n",
    "]\n",
    "llm.invoke(messages)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "a580a1e4-11a3-4277-bfba-bfb414ac7201",
   "metadata": {},
   "source": [
    "## Chaining\n",
    "\n",
    "We can [chain](/docs/how_to/sequence/) our model with a prompt template like so:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "dd0f4043-48bd-4245-8bdb-e7669666a277",
   "metadata": {},
   "outputs": [],
   "source": [
    "from langchain_core.prompts import ChatPromptTemplate\n",
    "\n",
    "prompt = ChatPromptTemplate(\n",
    "    [\n",
    "        (\n",
    "            \"system\",\n",
    "            \"You are a helpful assistant that translates {input_language} to {output_language}.\",\n",
    "        ),\n",
    "        (\"human\", \"{input}\"),\n",
    "    ]\n",
    ")\n",
    "\n",
    "chain = prompt | llm\n",
    "chain.invoke(\n",
    "    {\n",
    "        \"input_language\": \"English\",\n",
    "        \"output_language\": \"German\",\n",
    "        \"input\": \"I love programming.\",\n",
    "    }\n",
    ")"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "265f5d51-0a76-4808-8d13-ef598ee6e366",
   "metadata": {},
   "source": [
    "## API reference\n",
    "\n",


"metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "chat.invoke([system_message, human_message], temperature=0.7, max_tokens=10, top_p=0.95)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "> If you are going to place system prompt here, then it will override your system prompt that was fixed while deploying the application from the platform. "
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Native RAG Support with Prem Repositories\n",
    "\n",
    "Prem Repositories which allows users to upload documents (.txt, .pdf etc) and connect those repositories to the LLMs. You can think Prem repositories as native RAG, where each repository can be considered as a vector database. You can connect multiple repositories. You can learn more about repositories [here](https://docs.premai.io/get-started/repositories).\n",
    "\n",
    "Repositories are also supported in langchain premai. Here is how you can do it. "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 7,
   "metadata": {},
   "outputs": [],
   "source": [
    "query = \"Which models are used for dense retrieval\"\n",
    "repository_ids = [\n",
    "    1985,\n",
    "]\n",
    "repositories = dict(ids=repository_ids, similarity_threshold=0.3, limit=3)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "First we start by defining our repository with some repository ids. Make sure that the ids are valid repository ids. You can learn more about how to get the repository id [here](https://docs.premai.io/get-started/repositories). \n",
    "\n",
    "> Please note: Similar like `model_name` when you invoke the argument `repositories`, then you are potentially overriding the repositories connected in the launchpad. \n",
    "\n",
    "Now, we connect the repository with our chat object to invoke RAG based generations. "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 8,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Dense retrieval models typically include:\n",
      "\n",
      "1. **BERT-based Models**: Such as DPR (Dense Passage Retrieval) which uses BERT for encoding queries and passages.\n",
      "2. **ColBERT**: A model that combines BERT with late interaction mechanisms.\n",
      "3. **ANCE (Approximate Nearest Neighbor Negative Contrastive Estimation)**: Uses BERT and focuses on efficient retrieval.\n",
      "4. **TCT-ColBERT**: A variant of ColBERT that uses a two-tower\n",
      "{\n",
      "    \"document_chunks\": [\n",
      "        {\n",
      "            \"repository_id\": 1985,\n",
      "            \"document_id\": 1306,\n",
      "            \"chunk_id\": 173899,\n",
      "            \"document_name\": \"[D] Difference between sparse and dense informati\\u2026\",\n",
      "            \"similarity_score\": 0.3209080100059509,\n",
      "            \"content\": \"with the difference or anywhere\\nwhere I can read about it?\\n\\n\\n      17                  9\\n\\n\\n      u/ScotiabankCanada        \\u2022  Promoted\\n\\n\\n                       Accelerate your study permit process\\n                       with Scotiabank's Student GIC\\n                       Program. We're here to help you tur\\u2026\\n\\n\\n                       startright.scotiabank.com         Learn More\\n\\n\\n                            Add a Comment\\n\\n\\nSort by:   Best\\n\\n\\n      DinosParkour      \\u2022 1y ago\\n\\n\\n     Dense Retrieval (DR) m\"\n",
      "        }\n",
      "    ]\n",
      "}\n"
     ]
    }
   ],
   "source": [
    "import json\n",
    "\n",
    "response = chat.invoke(query, max_tokens=100, repositories=repositories)\n",
    "\n",
    "print(response.content)\n",
    "print(json.dumps(response.response_metadata, indent=4))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "> Ideally, you do not need to connect Repository IDs here to get Retrieval Augmented Generations. You can still get the same result if you have connected the repositories in prem platform. "
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Prem Templates\n",
    "\n",
    "Writing Prompt Templates can be super messy. Prompt templates are long, hard to manage, and must be continuously tweaked to improve and keep the same throughout the application. \n",
    "\n",
    "With **Prem**, writing and managing prompts can be super easy. The **_Templates_** tab inside the [launchpad](https://docs.premai.io/get-started/launchpad) helps you write as many prompts you need and use it inside the SDK to make your application running using those prompts. You can read more about Prompt Templates [here](https://docs.premai.io/get-started/prem-templates). \n",
    "\n",
    "To use Prem Templates natively with LangChain, you need to pass an id the `HumanMessage`. This id should be the name the variable of your prompt template. the `content` in `HumanMessage` should be the value of that variable. \n",
    "\n",
    "let's say for example, if your prompt template was this:\n",
    "\n",
    "```text\n",
    "Say hello to my name and say a feel-good quote\n",
    "from my age. My name is: {name} and age is {age}\n",
    "```\n",
    "\n",
    "So now your human_messages should look like:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 9,
   "metadata": {},
   "outputs": [],
   "source": [
    "human_messages = [\n",
    "    HumanMessage(content=\"Shawn\", id=\"name\"),\n",
    "    HumanMessage(content=\"22\", id=\"age\"),\n",
    "]"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "\n",
    "Pass this `human_messages` to ChatPremAI Client. Please note: Do not forget to\n",
    "pass the additional `template_id` to invoke generation with Prem Templates. If you are not aware of `template_id` you can learn more about that [in our docs](https://docs.premai.io/get-started/prem-templates). Here is an example:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "template_id = \"78069ce8-xxxxx-xxxxx-xxxx-xxx\"\n",
    "response = chat.invoke([human_messages], template_id=template_id)\n",
    "print(response.content)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Prem Template feature is available in streaming too. "
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Streaming\n",
    "\n",
    "In this section, let's see how we can stream tokens using langchain and PremAI. Here's how you do it. "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 17,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "It looks like your message got cut off. If you need information about Dense Retrieval (DR) or any other topic, please provide more details or clarify your question."
     ]
    }
   ],
   "source": [
    "import sys\n",
    "\n",
    "for chunk in chat.stream(\"hello how are you\"):\n",
    "    sys.stdout.write(chunk.content)\n",
    "    sys.stdout.flush()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Similar to above, if you want to override the system-prompt and the generation parameters, you need to add the following:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 18,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [