{
 "cells": [
  {
   "cell_type": "raw",
   "id": "afaf8039",
   "metadata": {},
   "source": [
    "---\n",
    "sidebar_label: Ollama\n",
    "---"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "e49f1e0d",
   "metadata": {},
   "source": [
    "# ChatOllama\n",
    "\n",
    "[Ollama](https://ollama.ai/) allows you to run open-source large language models, such as Llama 2, locally.\n",
    "\n",
    "Ollama bundles model weights, configuration, and data into a single package, defined by a Modelfile. \n",
    "\n",
    "It optimizes setup and configuration details, including GPU usage.\n",
    "\n",
    "For a complete list of supported models and model variants, see the [Ollama model library](https://github.com/jmorganca/ollama#model-library).\n",
    "\n",
    "## Overview\n",
    "### Integration details\n",
    "\n",
    "| Class | Package | Local | Serializable | [JS support](https://js.langchain.com/v0.2/docs/integrations/chat/ollama) | Package downloads | Package latest |\n",
    "| :--- | :--- | :---: | :---: |  :---: | :---: | :---: |\n",
    "| [ChatOllama](https://python.langchain.com/v0.2/api_reference/ollama/chat_models/langchain_ollama.chat_models.ChatOllama.html) | [langchain-ollama](https://python.langchain.com/v0.2/api_reference/ollama/index.html) | ✅ | ❌ | ✅ | ![PyPI - Downloads](https://img.shields.io/pypi/dm/langchain-ollama?style=flat-square&label=%20) | ![PyPI - Version](https://img.shields.io/pypi/v/langchain-ollama?style=flat-square&label=%20) |\n",
    "\n",
    "### Model features\n",
    "| [Tool calling](/docs/how_to/tool_calling/) | [Structured output](/docs/how_to/structured_output/) | JSON mode | [Image input](/docs/how_to/multimodal_inputs/) | Audio input | Video input | [Token-level streaming](/docs/how_to/chat_streaming/) | Native async | [Token usage](/docs/how_to/chat_token_usage_tracking/) | [Logprobs](/docs/how_to/logprobs/) |\n",
    "| :---: | :---: | :---: | :---: |  :---: | :---: | :---: | :---: | :---: | :---: |\n",
    "| ✅ | ❌ | ✅ | ❌ | ❌ | ❌ | ✅ | ✅ | ❌ | ❌ | \n",
    "\n",
    "## Setup\n",
    "\n",
    "First, follow [these instructions](https://github.com/jmorganca/ollama) to set up and run a local Ollama instance:\n",
    "\n",
    "* [Download](https://ollama.ai/download) and install Ollama onto the available supported platforms (including Windows Subsystem for Linux)\n",
    "* Fetch available LLM model via `ollama pull <name-of-model>`\n",
    "    * View a list of available models via the [model library](https://ollama.ai/library)\n",
    "    * e.g., `ollama pull llama3`\n",
    "* This will download the default tagged version of the model. Typically, the default points to the latest, smallest sized-parameter model.\n",
    "\n",
    "> On Mac, the models will be download to `~/.ollama/models`\n",
    "> \n",
    "> On Linux (or WSL), the models will be stored at `/usr/share/ollama/.ollama/models`\n",
    "\n",
    "* Specify the exact version of the model of interest as such `ollama pull vicuna:13b-v1.5-16k-q4_0` (View the [various tags for the `Vicuna`](https://ollama.ai/library/vicuna/tags) model in this instance)\n",
    "* To view all pulled models, use `ollama list`\n",
    "* To chat directly with a model from the command line, use `ollama run <name-of-model>`\n",
    "* View the [Ollama documentation](https://github.com/jmorganca/ollama) for more commands. Run `ollama help` in the terminal to see available commands too.\n"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "72ee0c4b-9764-423a-9dbf-95129e185210",
   "metadata": {},
   "source": [
    "If you want to get automated tracing of your model calls you can also set your [LangSmith](https://docs.smith.langchain.com/) API key by uncommenting below:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "a15d341e-3e26-4ca3-830b-5aab30ed66de",
   "metadata": {},
   "outputs": [],
   "source": [
    "# os.environ[\"LANGSMITH_API_KEY\"] = getpass.getpass(\"Enter your LangSmith API key: \")\n",
    "# os.environ[\"LANGSMITH_TRACING\"] = \"true\""
   ]
  },
  {
   "cell_type": "markdown",
   "id": "0730d6a1-c893-4840-9817-5e5251676d5d",
   "metadata": {},
   "source": [
    "### Installation\n",
    "\n",
    "The LangChain Ollama integration lives in the `langchain-ollama` package:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "652d6238-1f87-422a-b135-f5abbb8652fc",
   "metadata": {},
   "outputs": [],
   "source": [
    "%pip install -qU langchain-ollama"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "a38cde65-254d-4219-a441-068766c0d4b5",
   "metadata": {},
   "source": [
    "## Instantiation\n",
    "\n",
    "Now we can instantiate our model object and generate chat completions:\n",
    "\n",
    "- TODO: Update model instantiation with relevant params."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 9,
   "id": "cb09c344-1836-4e0c-acf8-11d13ac1dbae",
   "metadata": {},
   "outputs": [],
   "source": [
    "from langchain_ollama import ChatOllama\n",
    "\n",
    "llm = ChatOllama(\n",
    "    model=\"llama3.1\",\n",
    "    temperature=0,\n",
    "    # other params...\n",
    ")"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "2b4f3e15",
   "metadata": {},
   "source": [
    "## Invocation"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 10,
   "id": "62e0dbc3",
   "metadata": {
    "tags": []
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "AIMessage(content='The translation of \"I love programming\" from English to French is:\\n\\n\"J\\'adore programmer.\"', response_metadata={'model': 'llama3.1', 'created_at': '2024-08-19T16:05:32.81965Z', 'message': {'role': 'assistant', 'content': ''}, 'done_reason': 'stop', 'done': True, 'total_duration': 2167842917, 'load_duration': 54222584, 'prompt_eval_count': 35, 'prompt_eval_duration': 893007000, 'eval_count': 22, 'eval_duration': 1218962000}, id='run-0863daa2-43bf-4a43-86cc-611b23eae466-0', usage_metadata={'input_tokens': 35, 'output_tokens': 22, 'total_tokens': 57})"
      ]
     },
     "execution_count": 10,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "from langchain_core.messages import AIMessage\n",
    "\n",
    "messages = [\n",
    "    (\n",
    "        \"system\",\n",
    "        \"You are a helpful assistant that translates English to French. Translate the user sentence.\",\n",
    "    ),\n",


{
 "cells": [
  {
   "cell_type": "raw",
   "id": "7320f16b",
   "metadata": {},
   "source": [
    "---\n",
    "sidebar_label: Llama 2 Chat\n",
    "---"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "90a1faf2",
   "metadata": {},
   "source": [
    "# Llama2Chat\n",
    "\n",
    "This notebook shows how to augment Llama-2 `LLM`s with the `Llama2Chat` wrapper to support the [Llama-2 chat prompt format](https://huggingface.co/blog/llama2#how-to-prompt-llama-2). Several `LLM` implementations in LangChain can be used as interface to Llama-2 chat models. These include [ChatHuggingFace](/docs/integrations/chat/huggingface), [LlamaCpp](/docs/tutorials/local_rag), [GPT4All](/docs/integrations/llms/gpt4all), ..., to mention a few examples. \n",
    "\n",
    "`Llama2Chat` is a generic wrapper that implements `BaseChatModel` and can therefore be used in applications as [chat model](/docs/how_to#chat-models). `Llama2Chat` converts a list of Messages into the [required chat prompt format](https://huggingface.co/blog/llama2#how-to-prompt-llama-2) and forwards the formatted prompt as `str` to the wrapped `LLM`."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 1,
   "id": "36c03540",
   "metadata": {},
   "outputs": [],
   "source": [
    "from langchain.chains import LLMChain\n",
    "from langchain.memory import ConversationBufferMemory\n",
    "from langchain_experimental.chat_models import Llama2Chat"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "5c76910f",
   "metadata": {},
   "source": [
    "For the chat application examples below, we'll use the following chat `prompt_template`:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 2,
   "id": "9bbfaf3a",
   "metadata": {},
   "outputs": [],
   "source": [
    "from langchain_core.messages import SystemMessage\n",
    "from langchain_core.prompts.chat import (\n",
    "    ChatPromptTemplate,\n",
    "    HumanMessagePromptTemplate,\n",
    "    MessagesPlaceholder,\n",
    ")\n",
    "\n",
    "template_messages = [\n",
    "    SystemMessage(content=\"You are a helpful assistant.\"),\n",
    "    MessagesPlaceholder(variable_name=\"chat_history\"),\n",
    "    HumanMessagePromptTemplate.from_template(\"{text}\"),\n",
    "]\n",
    "prompt_template = ChatPromptTemplate.from_messages(template_messages)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "2f3343b7",
   "metadata": {},
   "source": [
    "## Chat with Llama-2 via `HuggingFaceTextGenInference` LLM"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "2ff99380",
   "metadata": {},
   "source": [
    "A HuggingFaceTextGenInference LLM encapsulates access to a [text-generation-inference](https://github.com/huggingface/text-generation-inference) server. In the following example, the inference server serves a [meta-llama/Llama-2-13b-chat-hf](https://huggingface.co/meta-llama/Llama-2-13b-chat-hf) model. It can be started locally with:\n",
    "\n",
    "```bash\n",
    "docker run \\\n",
    "  --rm \\\n",
    "  --gpus all \\\n",
    "  --ipc=host \\\n",
    "  -p 8080:80 \\\n",
    "  -v ~/.cache/huggingface/hub:/data \\\n",
    "  -e HF_API_TOKEN=${HF_API_TOKEN} \\\n",
    "  ghcr.io/huggingface/text-generation-inference:0.9 \\\n",
    "  --hostname 0.0.0.0 \\\n",
    "  --model-id meta-llama/Llama-2-13b-chat-hf \\\n",
    "  --quantize bitsandbytes \\\n",
    "  --num-shard 4\n",
    "```\n",
    "\n",
    "This works on a machine with 4 x RTX 3080ti cards, for example. Adjust the `--num_shard` value to the number of GPUs available. The `HF_API_TOKEN` environment variable holds the Hugging Face API token."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "238095fd",
   "metadata": {},
   "outputs": [],
   "source": [
    "# !pip3 install text-generation"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "79c4ace9",
   "metadata": {},
   "source": [
    "Create a `HuggingFaceTextGenInference` instance that connects to the local inference server and wrap it into `Llama2Chat`."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 4,
   "id": "7a9f6de2",
   "metadata": {},
   "outputs": [],
   "source": [
    "from langchain_community.llms import HuggingFaceTextGenInference\n",
    "\n",
    "llm = HuggingFaceTextGenInference(\n",
    "    inference_server_url=\"http://127.0.0.1:8080/\",\n",
    "    max_new_tokens=512,\n",
    "    top_k=50,\n",
    "    temperature=0.1,\n",
    "    repetition_penalty=1.03,\n",
    ")\n",
    "\n",
    "model = Llama2Chat(llm=llm)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "4f646a2b",
   "metadata": {},
   "source": [
    "Then you are ready to use the chat `model` together with `prompt_template` and conversation `memory` in an `LLMChain`."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 5,
   "id": "54b5d1d1",
   "metadata": {},
   "outputs": [],
   "source": [
    "memory = ConversationBufferMemory(memory_key=\"chat_history\", return_messages=True)\n",
    "chain = LLMChain(llm=model, prompt=prompt_template, memory=memory)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 6,
   "id": "e6717947",
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      " Sure, I'd be happy to help! Here are a few popular locations to consider visiting in Vienna:\n",
      "\n",
      "1. Schönbrunn Palace\n",
      "2. St. Stephen's Cathedral\n",
      "3. Hofburg Palace\n",
      "4. Belvedere Palace\n",
      "5. Prater Park\n",
      "6. Vienna State Opera\n",
      "7. Albertina Museum\n",
      "8. Museum of Natural History\n",
      "9. Kunsthistorisches Museum\n",
      "10. Ringstrasse\n"
     ]
    }
   ],
   "source": [
    "print(\n",
    "    chain.run(\n",
    "        text=\"What can I see in Vienna? Propose a few locations. Names only, no details.\"\n",
    "    )\n",
    ")"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 7,
   "id": "17bf10d5",
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      " Certainly! St. Stephen's Cathedral (Stephansdom) is one of the most recognizable landmarks in Vienna and a must-see attraction for visitors. This stunning Gothic cathedral is located in the heart of the city and is known for its intricate stone carvings, colorful stained glass windows, and impressive dome.\n",
      "\n",


{
 "cells": [
  {
   "cell_type": "raw",
   "id": "afaf8039",
   "metadata": {},
   "source": [
    "---\n",
    "sidebar_label: Azure OpenAI\n",
    "---"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "e49f1e0d",
   "metadata": {},
   "source": [
    "# AzureChatOpenAI\n",
    "\n",
    "This guide will help you get started with AzureOpenAI [chat models](/docs/concepts/#chat-models). For detailed documentation of all AzureChatOpenAI features and configurations head to the [API reference](https://python.langchain.com/api_reference/openai/chat_models/langchain_openai.chat_models.azure.AzureChatOpenAI.html).\n",
    "\n",
    "Azure OpenAI has several chat models. You can find information about their latest models and their costs, context windows, and supported input types in the [Azure docs](https://learn.microsoft.com/en-us/azure/ai-services/openai/concepts/models).\n",
    "\n",
    ":::info Azure OpenAI vs OpenAI\n",
    "\n",
    "Azure OpenAI refers to OpenAI models hosted on the [Microsoft Azure platform](https://azure.microsoft.com/en-us/products/ai-services/openai-service). OpenAI also provides its own model APIs. To access OpenAI services directly, use the [ChatOpenAI integration](/docs/integrations/chat/openai/).\n",
    "\n",
    ":::\n",
    "\n",
    "## Overview\n",
    "### Integration details\n",
    "\n",
    "| Class | Package | Local | Serializable | [JS support](https://js.langchain.com/docs/integrations/chat/azure) | Package downloads | Package latest |\n",
    "| :--- | :--- | :---: | :---: |  :---: | :---: | :---: |\n",
    "| [AzureChatOpenAI](https://python.langchain.com/api_reference/openai/chat_models/langchain_openai.chat_models.azure.AzureChatOpenAI.html) | [langchain-openai](https://python.langchain.com/api_reference/openai/index.html) | ❌ | beta | ✅ | ![PyPI - Downloads](https://img.shields.io/pypi/dm/langchain-openai?style=flat-square&label=%20) | ![PyPI - Version](https://img.shields.io/pypi/v/langchain-openai?style=flat-square&label=%20) |\n",
    "\n",
    "### Model features\n",
    "| [Tool calling](/docs/how_to/tool_calling) | [Structured output](/docs/how_to/structured_output/) | JSON mode | [Image input](/docs/how_to/multimodal_inputs/) | Audio input | Video input | [Token-level streaming](/docs/how_to/chat_streaming/) | Native async | [Token usage](/docs/how_to/chat_token_usage_tracking/) | [Logprobs](/docs/how_to/logprobs/) |\n",
    "| :---: | :---: | :---: | :---: |  :---: | :---: | :---: | :---: | :---: | :---: |\n",
    "| ✅ | ✅ | ✅ | ✅ | ❌ | ❌ | ✅ | ✅ | ✅ | ✅ | \n",
    "\n",
    "## Setup\n",
    "\n",
    "To access AzureOpenAI models you'll need to create an Azure account, create a deployment of an Azure OpenAI model, get the name and endpoint for your deployment, get an Azure OpenAI API key, and install the `langchain-openai` integration package.\n",
    "\n",
    "### Credentials\n",
    "\n",
    "Head to the [Azure docs](https://learn.microsoft.com/en-us/azure/ai-services/openai/chatgpt-quickstart?tabs=command-line%2Cpython-new&pivots=programming-language-python) to create your deployment and generate an API key. Once you've done this set the AZURE_OPENAI_API_KEY and AZURE_OPENAI_ENDPOINT environment variables:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "433e8d2b-9519-4b49-b2c4-7ab65b046c94",
   "metadata": {},
   "outputs": [],
   "source": [
    "import getpass\n",
    "import os\n",
    "\n",
    "if \"AZURE_OPENAI_API_KEY\" not in os.environ:\n",
    "    os.environ[\"AZURE_OPENAI_API_KEY\"] = getpass.getpass(\n",
    "        \"Enter your AzureOpenAI API key: \"\n",
    "    )\n",
    "os.environ[\"AZURE_OPENAI_ENDPOINT\"] = \"https://YOUR-ENDPOINT.openai.azure.com/\""
   ]
  },
  {
   "cell_type": "markdown",
   "id": "72ee0c4b-9764-423a-9dbf-95129e185210",
   "metadata": {},
   "source": [
    "If you want to get automated tracing of your model calls you can also set your [LangSmith](https://docs.smith.langchain.com/) API key by uncommenting below:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "a15d341e-3e26-4ca3-830b-5aab30ed66de",
   "metadata": {},
   "outputs": [],
   "source": [
    "# os.environ[\"LANGSMITH_API_KEY\"] = getpass.getpass(\"Enter your LangSmith API key: \")\n",
    "# os.environ[\"LANGSMITH_TRACING\"] = \"true\""
   ]
  },
  {
   "cell_type": "markdown",
   "id": "0730d6a1-c893-4840-9817-5e5251676d5d",
   "metadata": {},
   "source": [
    "### Installation\n",
    "\n",
    "The LangChain AzureOpenAI integration lives in the `langchain-openai` package:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "652d6238-1f87-422a-b135-f5abbb8652fc",
   "metadata": {},
   "outputs": [],
   "source": [
    "%pip install -qU langchain-openai"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "a38cde65-254d-4219-a441-068766c0d4b5",
   "metadata": {},
   "source": [
    "## Instantiation\n",
    "\n",
    "Now we can instantiate our model object and generate chat completions.\n",
    "- Replace `azure_deployment` with the name of your deployment,\n",
    "- You can find the latest supported `api_version` here: https://learn.microsoft.com/en-us/azure/ai-services/openai/reference."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 2,
   "id": "cb09c344-1836-4e0c-acf8-11d13ac1dbae",
   "metadata": {},
   "outputs": [],
   "source": [
    "from langchain_openai import AzureChatOpenAI\n",
    "\n",
    "llm = AzureChatOpenAI(\n",
    "    azure_deployment=\"gpt-35-turbo\",  # or your deployment\n",
    "    api_version=\"2023-06-01-preview\",  # or your api version\n",
    "    temperature=0,\n",
    "    max_tokens=None,\n",
    "    timeout=None,\n",
    "    max_retries=2,\n",
    "    # other params...\n",
    ")"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "2b4f3e15",
   "metadata": {},
   "source": [
    "## Invocation"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 3,
   "id": "62e0dbc3",
   "metadata": {
    "tags": []
   },
   "outputs": [
    {
     "data": {
      "text/plain": [


"AIMessage(content=\"J'adore la programmation.\", response_metadata={'token_usage': {'completion_tokens': 8, 'prompt_tokens': 31, 'total_tokens': 39}, 'model_name': 'gpt-35-turbo', 'system_fingerprint': None, 'prompt_filter_results': [{'prompt_index': 0, 'content_filter_results': {'hate': {'filtered': False, 'severity': 'safe'}, 'self_harm': {'filtered': False, 'severity': 'safe'}, 'sexual': {'filtered': False, 'severity': 'safe'}, 'violence': {'filtered': False, 'severity': 'safe'}}}], 'finish_reason': 'stop', 'logprobs': None, 'content_filter_results': {'hate': {'filtered': False, 'severity': 'safe'}, 'self_harm': {'filtered': False, 'severity': 'safe'}, 'sexual': {'filtered': False, 'severity': 'safe'}, 'violence': {'filtered': False, 'severity': 'safe'}}}, id='run-bea4b46c-e3e1-4495-9d3a-698370ad963d-0', usage_metadata={'input_tokens': 31, 'output_tokens': 8, 'total_tokens': 39})"
      ]
     },
     "execution_count": 3,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "messages = [\n",
    "    (\n",
    "        \"system\",\n",
    "        \"You are a helpful assistant that translates English to French. Translate the user sentence.\",\n",
    "    ),\n",
    "    (\"human\", \"I love programming.\"),\n",
    "]\n",
    "ai_msg = llm.invoke(messages)\n",
    "ai_msg"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 4,
   "id": "d86145b3-bfef-46e8-b227-4dda5c9c2705",
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "J'adore la programmation.\n"
     ]
    }
   ],
   "source": [
    "print(ai_msg.content)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "18e2bfc0-7e78-4528-a73f-499ac150dca8",
   "metadata": {},
   "source": [
    "## Chaining\n",
    "\n",
    "We can [chain](/docs/how_to/sequence/) our model with a prompt template like so:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 5,
   "id": "e197d1d7-a070-4c96-9f8a-a0e86d046e0b",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "AIMessage(content='Ich liebe das Programmieren.', response_metadata={'token_usage': {'completion_tokens': 6, 'prompt_tokens': 26, 'total_tokens': 32}, 'model_name': 'gpt-35-turbo', 'system_fingerprint': None, 'prompt_filter_results': [{'prompt_index': 0, 'content_filter_results': {'hate': {'filtered': False, 'severity': 'safe'}, 'self_harm': {'filtered': False, 'severity': 'safe'}, 'sexual': {'filtered': False, 'severity': 'safe'}, 'violence': {'filtered': False, 'severity': 'safe'}}}], 'finish_reason': 'stop', 'logprobs': None, 'content_filter_results': {'hate': {'filtered': False, 'severity': 'safe'}, 'self_harm': {'filtered': False, 'severity': 'safe'}, 'sexual': {'filtered': False, 'severity': 'safe'}, 'violence': {'filtered': False, 'severity': 'safe'}}}, id='run-cbc44038-09d3-40d4-9da2-c5910ee636ca-0', usage_metadata={'input_tokens': 26, 'output_tokens': 6, 'total_tokens': 32})"
      ]
     },
     "execution_count": 5,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "from langchain_core.prompts import ChatPromptTemplate\n",
    "\n",
    "prompt = ChatPromptTemplate.from_messages(\n",
    "    [\n",
    "        (\n",
    "            \"system\",\n",
    "            \"You are a helpful assistant that translates {input_language} to {output_language}.\",\n",
    "        ),\n",
    "        (\"human\", \"{input}\"),\n",
    "    ]\n",
    ")\n",
    "\n",
    "chain = prompt | llm\n",
    "chain.invoke(\n",
    "    {\n",
    "        \"input_language\": \"English\",\n",
    "        \"output_language\": \"German\",\n",
    "        \"input\": \"I love programming.\",\n",
    "    }\n",
    ")"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "d1ee55bc-ffc8-4cfa-801c-993953a08cfd",
   "metadata": {},
   "source": [
    "## Specifying model version\n",
    "\n",
    "Azure OpenAI responses contain `model_name` response metadata property, which is name of the model used to generate the response. However unlike native OpenAI responses, it does not contain the specific version of the model, which is set on the deployment in Azure. E.g. it does not distinguish between `gpt-35-turbo-0125` and `gpt-35-turbo-0301`. This makes it tricky to know which version of the model was used to generate the response, which as result can lead to e.g. wrong total cost calculation with `OpenAICallbackHandler`.\n",
    "\n",
    "To solve this problem, you can pass `model_version` parameter to `AzureChatOpenAI` class, which will be added to the model name in the llm output. This way you can easily distinguish between different versions of the model."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "04b36e75-e8b7-4721-899e-76301ac2ecd9",
   "metadata": {},
   "outputs": [],
   "source": [
    "%pip install -qU langchain-community"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 6,
   "id": "2ca02d23-60d0-43eb-8d04-070f61f8fefd",
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Total Cost (USD): $0.000063\n"
     ]
    }
   ],
   "source": [
    "from langchain_community.callbacks import get_openai_callback\n",
    "\n",
    "with get_openai_callback() as cb:\n",
    "    llm.invoke(messages)\n",
    "    print(\n",
    "        f\"Total Cost (USD): ${format(cb.total_cost, '.6f')}\"\n",
    "    )  # without specifying the model version, flat-rate 0.002 USD per 1k input and output tokens is used"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 7,
   "id": "e1b07ae2-3de7-44bd-bfdc-b76f4ba45a35",
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Total Cost (USD): $0.000074\n"
     ]
    }
   ],
   "source": [
    "llm_0301 = AzureChatOpenAI(\n",
    "    azure_deployment=\"gpt-35-turbo\",  # or your deployment\n",
    "    api_version=\"2023-06-01-preview\",  # or your api version\n",
    "    model_version=\"0301\",\n",
    ")\n",
    "with get_openai_callback() as cb:\n",
    "    llm_0301.invoke(messages)\n",
    "    print(f\"Total Cost (USD): ${format(cb.total_cost, '.6f')}\")"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "3a5bb5ca-c3ae-4a58-be67-2cd18574b9a3",
   "metadata": {},
   "source": [
    "## API reference\n",
    "\n",


{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "\n",
    "  \n",
    ""
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# UpTrain\n",
    "\n",
    "> UpTrain [[github](https://github.com/uptrain-ai/uptrain) || [website](https://uptrain.ai/) || [docs](https://docs.uptrain.ai/getting-started/introduction)] is an open-source platform to evaluate and improve LLM applications. It provides grades for 20+ preconfigured checks (covering language, code, embedding use cases), performs root cause analyses on instances of failure cases and provides guidance for resolving them."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## UpTrain Callback Handler\n",
    "\n",
    "This notebook showcases the UpTrain callback handler seamlessly integrating into your pipeline, facilitating diverse evaluations. We have chosen a few evaluations that we deemed apt for evaluating the chains. These evaluations run automatically, with results displayed in the output. More details on UpTrain's evaluations can be found [here](https://github.com/uptrain-ai/uptrain?tab=readme-ov-file#pre-built-evaluations-we-offer-). \n",
    "\n",
    "Selected retievers from Langchain are highlighted for demonstration:\n",
    "\n",
    "### 1. **Vanilla RAG**:\n",
    "RAG plays a crucial role in retrieving context and generating responses. To ensure its performance and response quality, we conduct the following evaluations:\n",
    "\n",
    "- **[Context Relevance](https://docs.uptrain.ai/predefined-evaluations/context-awareness/context-relevance)**: Determines if the context extracted from the query is relevant to the response.\n",
    "- **[Factual Accuracy](https://docs.uptrain.ai/predefined-evaluations/context-awareness/factual-accuracy)**: Assesses if the LLM is hallcuinating or providing incorrect information.\n",
    "- **[Response Completeness](https://docs.uptrain.ai/predefined-evaluations/response-quality/response-completeness)**: Checks if the response contains all the information requested by the query.\n",
    "\n",
    "### 2. **Multi Query Generation**:\n",
    "MultiQueryRetriever creates multiple variants of a question having a similar meaning to the original question. Given the complexity, we include the previous evaluations and add:\n",
    "\n",
    "- **[Multi Query Accuracy](https://docs.uptrain.ai/predefined-evaluations/query-quality/multi-query-accuracy)**: Assures that the multi-queries generated mean the same as the original query.\n",
    "\n",
    "### 3. **Context Compression and Reranking**:\n",
    "Re-ranking involves reordering nodes based on relevance to the query and choosing top n nodes. Since the number of nodes can reduce once the re-ranking is complete, we perform the following evaluations:\n",
    "\n",
    "- **[Context Reranking](https://docs.uptrain.ai/predefined-evaluations/context-awareness/context-reranking)**: Checks if the order of re-ranked nodes is more relevant to the query than the original order.\n",
    "- **[Context Conciseness](https://docs.uptrain.ai/predefined-evaluations/context-awareness/context-conciseness)**: Examines whether the reduced number of nodes still provides all the required information.\n",
    "\n",
    "These evaluations collectively ensure the robustness and effectiveness of the RAG, MultiQueryRetriever, and the Reranking process in the chain."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Install Dependencies"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 1,
   "metadata": {},
   "outputs": [
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "huggingface/tokenizers: The current process just got forked, after parallelism has already been used. Disabling parallelism to avoid deadlocks...\n",
      "To disable this warning, you can either:\n",
      "\t- Avoid using `tokenizers` before the fork if possible\n",
      "\t- Explicitly set the environment variable TOKENIZERS_PARALLELISM=(true | false)\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "\u001b[33mWARNING: There was an error checking the latest version of pip.\u001b[0m\u001b[33m\n",
      "\u001b[0mNote: you may need to restart the kernel to use updated packages.\n"
     ]
    }
   ],
   "source": [
    "%pip install -qU langchain langchain_openai langchain-community uptrain faiss-cpu flashrank"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "NOTE: that you can also install `faiss-gpu` instead of `faiss-cpu` if you want to use the GPU enabled version of the library."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Import Libraries"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 2,
   "metadata": {},
   "outputs": [],
   "source": [
    "from getpass import getpass\n",
    "\n",
    "from langchain.chains import RetrievalQA\n",
    "from langchain.retrievers import ContextualCompressionRetriever\n",
    "from langchain.retrievers.document_compressors import FlashrankRerank\n",
    "from langchain.retrievers.multi_query import MultiQueryRetriever\n",
    "from langchain_community.callbacks.uptrain_callback import UpTrainCallbackHandler\n",
    "from langchain_community.document_loaders import TextLoader\n",
    "from langchain_community.vectorstores import FAISS\n",
    "from langchain_core.output_parsers.string import StrOutputParser\n",
    "from langchain_core.prompts.chat import ChatPromptTemplate\n",
    "from langchain_core.runnables.passthrough import RunnablePassthrough\n",
    "from langchain_openai import ChatOpenAI, OpenAIEmbeddings\n",
    "from langchain_text_splitters import (\n",
    "    RecursiveCharacterTextSplitter,\n",
    ")"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Load the documents"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 3,
   "metadata": {},
   "outputs": [],
   "source": [
    "loader = TextLoader(\"../../how_to/state_of_the_union.txt\")\n",
    "documents = loader.load()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Split the document into chunks"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 4,
   "metadata": {},
   "outputs": [],
   "source": [
    "text_splitter = RecursiveCharacterTextSplitter(chunk_size=1000, chunk_overlap=0)\n",
    "chunks = text_splitter.split_documents(documents)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Create the retriever"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 5,
   "metadata": {},
   "outputs": [],
   "source": [
    "embeddings = OpenAIEmbeddings()\n",
    "db = FAISS.from_documents(chunks, embeddings)\n",
    "retriever = db.as_retriever()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Define the LLM"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 6,
   "metadata": {},
   "outputs": [],
   "source": [
    "llm = ChatOpenAI(temperature=0, model=\"gpt-4\")"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Setup\n",
    "\n",
    "UpTrain provides you with:\n",
    "1. Dashboards with advanced drill-down and filtering options\n",
    "1. Insights and common topics among failing cases\n",
    "1. Observability and real-time monitoring of production data\n",
    "1. Regression testing via seamless integration with your CI/CD pipelines\n",
    "\n",
    "You can choose between the following options for evaluating using UpTrain:\n",


# C Transformers

This page covers how to use the [C Transformers](https://github.com/marella/ctransformers) library within LangChain.
It is broken into two parts: installation and setup, and then references to specific C Transformers wrappers.

## Installation and Setup

- Install the Python package with `pip install ctransformers`
- Download a supported [GGML model](https://huggingface.co/TheBloke) (see [Supported Models](https://github.com/marella/ctransformers#supported-models))

## Wrappers

### LLM

There exists a CTransformers LLM wrapper, which you can access with:

```python
from langchain_community.llms import CTransformers
```

It provides a unified interface for all models:

```python
llm = CTransformers(model='/path/to/ggml-gpt-2.bin', model_type='gpt2')

print(llm.invoke('AI is going to'))
```

If you are getting `illegal instruction` error, try using `lib='avx'` or `lib='basic'`:

```py
llm = CTransformers(model='/path/to/ggml-gpt-2.bin', model_type='gpt2', lib='avx')
```

It can be used with models hosted on the Hugging Face Hub:

```py
llm = CTransformers(model='marella/gpt-2-ggml')
```

If a model repo has multiple model files (`.bin` files), specify a model file using:

```py
llm = CTransformers(model='marella/gpt-2-ggml', model_file='ggml-model.bin')
```

Additional parameters can be passed using the `config` parameter:

```py
config = {'max_new_tokens': 256, 'repetition_penalty': 1.1}

llm = CTransformers(model='marella/gpt-2-ggml', config=config)
```

See [Documentation](https://github.com/marella/ctransformers#config) for a list of available parameters.

For a more detailed walkthrough of this, see [this notebook](/docs/integrations/llms/ctransformers).


# MongoDB Atlas

>[MongoDB Atlas](https://www.mongodb.com/docs/atlas/) is a fully-managed cloud
> database available in AWS, Azure, and GCP.  It now has support for native
> Vector Search on the MongoDB document data.

## Installation and Setup

See [detail configuration instructions](/docs/integrations/vectorstores/mongodb_atlas).

We need to install `langchain-mongodb` python package.

```bash
pip install langchain-mongodb
```

## Vector Store

See a [usage example](/docs/integrations/vectorstores/mongodb_atlas).

```python
from langchain_mongodb import MongoDBAtlasVectorSearch
```

## Retrievers

### Full Text Search Retriever

>`Hybrid Search Retriever` performs full-text searches using
> Lucene’s standard (`BM25`) analyzer.

```python
from langchain_mongodb.retrievers import MongoDBAtlasFullTextSearchRetriever
```

### Hybrid Search Retriever

>`Hybrid Search Retriever` combines vector and full-text searches weighting
> them the via `Reciprocal Rank Fusion` (`RRF`) algorithm.

```python
from langchain_mongodb.retrievers import MongoDBAtlasHybridSearchRetriever
```

## Model Caches

### MongoDBCache

An abstraction to store a simple cache in MongoDB. This does not use Semantic Caching, nor does it require an index to be made on the collection before generation.

To import this cache:
```python
from langchain_mongodb.cache import MongoDBCache
```

To use this cache with your LLMs:
```python
from langchain_core.globals import set_llm_cache

# use any embedding provider...
from tests.integration_tests.vectorstores.fake_embeddings import FakeEmbeddings

mongodb_atlas_uri = "<YOUR_CONNECTION_STRING>"
COLLECTION_NAME="<YOUR_CACHE_COLLECTION_NAME>"
DATABASE_NAME="<YOUR_DATABASE_NAME>"

set_llm_cache(MongoDBCache(
    connection_string=mongodb_atlas_uri,
    collection_name=COLLECTION_NAME,
    database_name=DATABASE_NAME,
))
```

### MongoDBAtlasSemanticCache
Semantic caching allows users to retrieve cached prompts based on semantic similarity between the user input and previously cached results. Under the hood it blends MongoDBAtlas as both a cache and a vectorstore.
The MongoDBAtlasSemanticCache inherits from `MongoDBAtlasVectorSearch` and needs an Atlas Vector Search Index defined to work. Please look at the [usage example](/docs/integrations/vectorstores/mongodb_atlas) on how to set up the index.

To import this cache:
```python
from langchain_mongodb.cache import MongoDBAtlasSemanticCache
```

To use this cache with your LLMs:
```python
from langchain_core.globals import set_llm_cache

# use any embedding provider...
from tests.integration_tests.vectorstores.fake_embeddings import FakeEmbeddings

mongodb_atlas_uri = "<YOUR_CONNECTION_STRING>"
COLLECTION_NAME="<YOUR_CACHE_COLLECTION_NAME>"
DATABASE_NAME="<YOUR_DATABASE_NAME>"

set_llm_cache(MongoDBAtlasSemanticCache(
    embedding=FakeEmbeddings(),
    connection_string=mongodb_atlas_uri,
    collection_name=COLLECTION_NAME,
    database_name=DATABASE_NAME,
))
```
``


---
keywords: [openai]
---

# OpenAI

All functionality related to OpenAI

>[OpenAI](https://en.wikipedia.org/wiki/OpenAI) is American artificial intelligence (AI) research laboratory
> consisting of the non-profit `OpenAI Incorporated`
> and its for-profit subsidiary corporation `OpenAI Limited Partnership`.
> `OpenAI` conducts AI research with the declared intention of promoting and developing a friendly AI.
> `OpenAI` systems run on an `Azure`-based supercomputing platform from `Microsoft`.

>The [OpenAI API](https://platform.openai.com/docs/models) is powered by a diverse set of models with different capabilities and price points.
>
>[ChatGPT](https://chat.openai.com) is the Artificial Intelligence (AI) chatbot developed by `OpenAI`.

## Installation and Setup

Install the integration package with
```bash
pip install langchain-openai
```

Get an OpenAI api key and set it as an environment variable (`OPENAI_API_KEY`)

## Chat model

See a [usage example](/docs/integrations/chat/openai).

```python
from langchain_openai import ChatOpenAI
```

If you are using a model hosted on `Azure`, you should use different wrapper for that:
```python
from langchain_openai import AzureChatOpenAI
```
For a more detailed walkthrough of the `Azure` wrapper, see [here](/docs/integrations/chat/azure_chat_openai).

## LLM

See a [usage example](/docs/integrations/llms/openai).

```python
from langchain_openai import OpenAI
```

If you are using a model hosted on `Azure`, you should use different wrapper for that:
```python
from langchain_openai import AzureOpenAI
```
For a more detailed walkthrough of the `Azure` wrapper, see [here](/docs/integrations/llms/azure_openai).

## Embedding Model

See a [usage example](/docs/integrations/text_embedding/openai)

```python
from langchain_openai import OpenAIEmbeddings
```

## Document Loader

See a [usage example](/docs/integrations/document_loaders/chatgpt_loader).

```python
from langchain_community.document_loaders.chatgpt import ChatGPTLoader
```

## Retriever

See a [usage example](/docs/integrations/retrievers/chatgpt-plugin).

```python
from langchain.retrievers import ChatGPTPluginRetriever
```

## Tools

### Dall-E Image Generator

>[OpenAI Dall-E](https://openai.com/dall-e-3) are text-to-image models developed by `OpenAI`
> using deep learning methodologies to generate digital images from natural language descriptions,
> called "prompts".

See a [usage example](/docs/integrations/tools/dalle_image_generator).

```python
from langchain_community.utilities.dalle_image_generator import DallEAPIWrapper
```

## Adapter

See a [usage example](/docs/integrations/adapters/openai).

```python
from langchain.adapters import openai as lc_openai
```

## Tokenizer

There are several places you can use the `tiktoken` tokenizer. By default, it is used to count tokens
for OpenAI LLMs.

You can also use it to count tokens when splitting documents with
```python
from langchain.text_splitter import CharacterTextSplitter
CharacterTextSplitter.from_tiktoken_encoder(...)
```
For a more detailed walkthrough of this, see [this notebook](/docs/how_to/split_by_token/#tiktoken)

## Chain

See a [usage example](https://python.langchain.com/v0.1/docs/guides/productionization/safety/moderation).

```python
from langchain.chains import OpenAIModerationChain
```


# Zep
> Recall, understand, and extract data from chat histories. Power personalized AI experiences.

>[Zep](https://www.getzep.com) is a long-term memory service for AI Assistant apps.
> With Zep, you can provide AI assistants with the ability to recall past conversations, no matter how distant,
> while also reducing hallucinations, latency, and cost.

## How Zep works

Zep persists and recalls chat histories, and automatically generates summaries and other artifacts from these chat histories.
It also embeds messages and summaries, enabling you to search Zep for relevant context from past conversations.
Zep does all of this asynchronously, ensuring these operations don't impact your user's chat experience.
Data is persisted to database, allowing you to scale out when growth demands.

Zep also provides a simple, easy to use abstraction for document vector search called Document Collections.
This is designed to complement Zep's core memory features, but is not designed to be a general purpose vector database.

Zep allows you to be more intentional about constructing your prompt:
- automatically adding a few recent messages, with the number customized for your app;
- a summary of recent conversations prior to the messages above;
- and/or contextually relevant summaries or messages surfaced from the entire chat session.
- and/or relevant Business data from Zep Document Collections.

## What is Zep Cloud?
[Zep Cloud](https://www.getzep.com) is a managed service with Zep Open Source at its core.
In addition to Zep Open Source's memory management features, Zep Cloud offers:
- **Fact Extraction**: Automatically build fact tables from conversations, without having to define a data schema upfront.
- **Dialog Classification**: Instantly and accurately classify chat dialog. Understand user intent and emotion, segment users, and more. Route chains based on semantic context, and trigger events.
- **Structured Data Extraction**: Quickly extract business data from chat conversations using a schema you define. Understand what your Assistant should ask for next in order to complete its task.

## Zep Open Source
Zep offers an open source version with a self-hosted option.
Please refer to the [Zep Open Source](https://github.com/getzep/zep) repo for more information.
You can also find Zep Open Source compatible [Retriever](/docs/integrations/retrievers/zep_memorystore), [Vector Store](/docs/integrations/vectorstores/zep) and [Memory](/docs/integrations/memory/zep_memory) examples

## Zep Cloud Installation and Setup

[Zep Cloud Docs](https://help.getzep.com)

1. Install the Zep Cloud SDK:

```bash
pip install zep_cloud
```
or
```bash
poetry add zep_cloud
```

## Memory

Zep's Memory API persists your users' chat history and metadata to a [Session](https://help.getzep.com/chat-history-memory/sessions), enriches the memory, and
enables vector similarity search over historical chat messages and dialog summaries.

Zep offers several approaches to populating prompts with context from historical conversations.

### Perpetual Memory
This is the default memory type.
Salient facts from the dialog are extracted and stored in a Fact Table.
This is updated in real-time as new messages are added to the Session.
Every time you call the Memory API to get a Memory, Zep returns the Fact Table, the most recent messages (per your Message Window setting), and a summary of the most recent messages prior to the Message Window.
The combination of the Fact Table, summary, and the most recent messages in a prompts provides both factual context and nuance to the LLM.

### Summary Retriever Memory
Returns the most recent messages and a summary of past messages relevant to the current conversation,
enabling you to provide your Assistant with helpful context from past conversations

### Message Window Buffer Memory
Returns the most recent N messages from the current conversation.

Additionally, Zep enables vector similarity searches for Messages or Summaries stored within its system.

This feature lets you populate prompts with past conversations that are contextually similar to a specific query,
organizing the results by a similarity Score.

`ZepCloudChatMessageHistory` and `ZepCloudMemory` classes can be imported to interact with Zep Cloud APIs.

`ZepCloudChatMessageHistory` is compatible with `RunnableWithMessageHistory`.
```python
from langchain_community.chat_message_histories import ZepCloudChatMessageHistory
```

See a [Perpetual Memory Example here](/docs/integrations/memory/zep_cloud_chat_message_history).

You can use `ZepCloudMemory` together with agents that support Memory.
```python
from langchain_community.memory import ZepCloudMemory
```

See a [Memory RAG Example here](/docs/integrations/memory/zep_memory_cloud).

## Retriever

Zep's Memory Retriever is a LangChain Retriever that enables you to retrieve messages from a Zep Session and use them to construct your prompt.

The Retriever supports searching over both individual messages and summaries of conversations. The latter is useful for providing rich, but succinct context to the LLM as to relevant past conversations.

Zep's Memory Retriever supports both similarity search and [Maximum Marginal Relevance (MMR) reranking](https://help.getzep.com/working-with-search#how-zeps-mmr-re-ranking-works). MMR search is useful for ensuring that the retrieved messages are diverse and not too similar to each other

See a [usage example](/docs/integrations/retrievers/zep_cloud_memorystore).

```python
from langchain_community.retrievers import ZepCloudRetriever
```

## Vector store

Zep's [Document VectorStore API](https://help.getzep.com/document-collections) enables you to store and retrieve documents using vector similarity search. Zep doesn't require you to understand
distance functions, types of embeddings, or indexing best practices. You just pass in your chunked documents, and Zep handles the rest.

Zep supports both similarity search and [Maximum Marginal Relevance (MMR) reranking](https://help.getzep.com/working-with-search#how-zeps-mmr-re-ranking-works).
MMR search is useful for ensuring that the retrieved documents are diverse and not too similar to each other.

```python
from langchain_community.vectorstores import ZepCloudVectorStore
```

See a [usage example](/docs/integrations/vectorstores/zep_cloud).


# Log10

This page covers how to use the [Log10](https://log10.io) within LangChain.

## What is Log10?

Log10 is an [open-source](https://github.com/log10-io/log10) proxiless LLM data management and application development platform that lets you log, debug and tag your Langchain calls.

## Quick start

1. Create your free account at [log10.io](https://log10.io)
2. Add your `LOG10_TOKEN` and `LOG10_ORG_ID` from the Settings and Organization tabs respectively as environment variables.
3. Also add `LOG10_URL=https://log10.io` and your usual LLM API key: for e.g. `OPENAI_API_KEY` or `ANTHROPIC_API_KEY` to your environment

## How to enable Log10 data management for Langchain

Integration with log10 is a simple one-line `log10_callback` integration as shown below:

```python
from langchain_openai import ChatOpenAI
from langchain_core.messages import HumanMessage

from log10.langchain import Log10Callback
from log10.llm import Log10Config

log10_callback = Log10Callback(log10_config=Log10Config())

messages = [
    HumanMessage(content="You are a ping pong machine"),
    HumanMessage(content="Ping?"),
]

llm = ChatOpenAI(model="gpt-3.5-turbo", callbacks=[log10_callback])
```

[Log10 + Langchain + Logs docs](https://github.com/log10-io/log10/blob/main/logging.md#langchain-logger)

[More details + screenshots](https://log10.io/docs/observability/logs) including instructions for self-hosting logs

## How to use tags with Log10

```python
from langchain_openai import OpenAI
from langchain_community.chat_models import ChatAnthropic
from langchain_openai import ChatOpenAI
from langchain_core.messages import HumanMessage

from log10.langchain import Log10Callback
from log10.llm import Log10Config

log10_callback = Log10Callback(log10_config=Log10Config())

messages = [
    HumanMessage(content="You are a ping pong machine"),
    HumanMessage(content="Ping?"),
]

llm = ChatOpenAI(model="gpt-3.5-turbo", callbacks=[log10_callback], temperature=0.5, tags=["test"])
completion = llm.predict_messages(messages, tags=["foobar"])
print(completion)

llm = ChatAnthropic(model="claude-2", callbacks=[log10_callback], temperature=0.7, tags=["baz"])
llm.predict_messages(messages)
print(completion)

llm = OpenAI(model_name="gpt-3.5-turbo-instruct", callbacks=[log10_callback], temperature=0.5)
completion = llm.predict("You are a ping pong machine.\nPing?\n")
print(completion)
```

You can also intermix direct OpenAI calls and Langchain LLM calls:

```python
import os
from log10.load import log10, log10_session
import openai
from langchain_openai import OpenAI

log10(openai)

with log10_session(tags=["foo", "bar"]):
    # Log a direct OpenAI call
    response = openai.Completion.create(
        model="text-ada-001",
        prompt="Where is the Eiffel Tower?",
        temperature=0,
        max_tokens=1024,
        top_p=1,
        frequency_penalty=0,
        presence_penalty=0,
    )
    print(response)

    # Log a call via Langchain
    llm = OpenAI(model_name="text-ada-001", temperature=0.5)
    response = llm.predict("You are a ping pong machine.\nPing?\n")
    print(response)
```

## How to debug Langchain calls

[Example of debugging](https://log10.io/docs/observability/prompt_chain_debugging)

[More Langchain examples](https://github.com/log10-io/log10/tree/main/examples#langchain)


# LangChain Decorators ✨

~~~
Disclaimer: `LangChain decorators` is not created by the LangChain team and is not supported by it.
~~~

>`LangChain decorators` is a layer on the top of LangChain that provides syntactic sugar 🍭 for writing custom langchain prompts and chains
>
>For Feedback, Issues, Contributions - please raise an issue here:
>[ju-bezdek/langchain-decorators](https://github.com/ju-bezdek/langchain-decorators)

Main principles and benefits:

- more `pythonic` way of writing code
- write multiline prompts that won't break your code flow with indentation
- making use of IDE in-built support for **hinting**, **type checking** and **popup with docs** to quickly peek in the function to see the prompt, parameters it consumes etc.
- leverage all the power of 🦜🔗 LangChain ecosystem
- adding support for **optional parameters**
- easily share parameters between the prompts by binding them to one class

Here is a simple example of a code written with **LangChain Decorators ✨**

``` python

@llm_prompt
def write_me_short_post(topic:str, platform:str="twitter", audience:str = "developers")->str:
    """
    Write me a short header for my post about {topic} for {platform} platform.
    It should be for {audience} audience.
    (Max 15 words)
    """
    return

# run it naturally
write_me_short_post(topic="starwars")
# or
write_me_short_post(topic="starwars", platform="redit")
```

# Quick start
## Installation
```bash
pip install langchain_decorators
```

## Examples

Good idea on how to start is to review the examples here:
 - [jupyter notebook](https://github.com/ju-bezdek/langchain-decorators/blob/main/example_notebook.ipynb)
 - [colab notebook](https://colab.research.google.com/drive/1no-8WfeP6JaLD9yUtkPgym6x0G9ZYZOG#scrollTo=N4cf__D0E2Yk)

# Defining other parameters
Here we are just marking a function as a prompt with `llm_prompt` decorator, turning it effectively into a LLMChain. Instead of running it

Standard LLMchain takes much more init parameter than just inputs_variables and prompt... here is this implementation detail hidden in the decorator.
Here is how it works:

1. Using **Global settings**:

``` python
# define global settings for all prompty (if not set - chatGPT is the current default)
from langchain_decorators import GlobalSettings

GlobalSettings.define_settings(
    default_llm=ChatOpenAI(temperature=0.0), this is default... can change it here globally
    default_streaming_llm=ChatOpenAI(temperature=0.0,streaming=True), this is default... can change it here for all ... will be used for streaming
)
```

2. Using predefined **prompt types**

``` python
#You can change the default prompt types
from langchain_decorators import PromptTypes, PromptTypeSettings

PromptTypes.AGENT_REASONING.llm = ChatOpenAI()

# Or you can just define your own ones:
class MyCustomPromptTypes(PromptTypes):
    GPT4=PromptTypeSettings(llm=ChatOpenAI(model="gpt-4"))

@llm_prompt(prompt_type=MyCustomPromptTypes.GPT4)
def write_a_complicated_code(app_idea:str)->str:
    ...

```

3.  Define the settings **directly in the decorator**

``` python
from langchain_openai import OpenAI

@llm_prompt(
    llm=OpenAI(temperature=0.7),
    stop_tokens=["\nObservation"],
    ...
    )
def creative_writer(book_title:str)->str:
    ...
```

## Passing a memory and/or callbacks:

To pass any of these, just declare them in the function (or use kwargs to pass anything)

```python

@llm_prompt()
async def write_me_short_post(topic:str, platform:str="twitter", memory:SimpleMemory = None):
    """
    {history_key}
    Write me a short header for my post about {topic} for {platform} platform.
    It should be for {audience} audience.
    (Max 15 words)
    """
    pass

await write_me_short_post(topic="old movies")

```

# Simplified streaming

If we want to leverage streaming:
 - we need to define prompt as async function
 - turn on the streaming on the decorator, or we can define PromptType with streaming on
 - capture the stream using StreamingContext

This way we just mark which prompt should be streamed, not needing to tinker with what LLM should we use, passing around the creating and distribute streaming handler into particular part of our chain... just turn the streaming on/off on prompt/prompt type...

The streaming will happen only if we call it in streaming context ... there we can define a simple function to handle the stream

``` python
# this code example is complete and should run as it is

from langchain_decorators import StreamingContext, llm_prompt

# this will mark the prompt for streaming (useful if we want stream just some prompts in our app... but don't want to pass distribute the callback handlers)
# note that only async functions can be streamed (will get an error if it's not)
@llm_prompt(capture_stream=True)
async def write_me_short_post(topic:str, platform:str="twitter", audience:str = "developers"):
    """
    Write me a short header for my post about {topic} for {platform} platform.
    It should be for {audience} audience.
    (Max 15 words)
    """
    pass

# just an arbitrary  function to demonstrate the streaming... will be some websockets code in the real world
tokens=[]
def capture_stream_func(new_token:str):
    tokens.append(new_token)

# if we want to capture the stream, we need to wrap the execution into StreamingContext...
# this will allow us to capture the stream even if the prompt call is hidden inside higher level method
# only the prompts marked with capture_stream will be captured here
with StreamingContext(stream_to_stdout=True, callback=capture_stream_func):
    result = await run_prompt()
    print("Stream finished ... we can distinguish tokens thanks to alternating colors")

print("\nWe've captured",len(tokens),"tokens🎉\n")
print("Here is the result:")
print(result)
```

# Prompt declarations
By default the prompt is is the whole function docs, unless you mark your prompt

## Documenting your prompt

We can specify what part of our docs is the prompt definition, by specifying a code block with `<prompt>` language tag

``` python
@llm_prompt
def write_me_short_post(topic:str, platform:str="twitter", audience:str = "developers"):
    """
    Here is a good way to write a prompt as part of a function docstring, with additional documentation for devs.

    It needs to be a code block, marked as a `<prompt>` language
    ```<prompt>
    Write me a short header for my post about {topic} for {platform} platform.
    It should be for {audience} audience.
    (Max 15 words)
    ```

    Now only to code block above will be used as a prompt, and the rest of the docstring will be used as a description for developers.
    (It has also a nice benefit that IDE (like VS code) will display the prompt properly (not trying to parse it as markdown, and thus not showing new lines properly))
    """
    return
```

## Chat messages prompt

For chat models is very useful to define prompt as a set of message templates... here is how to do it:

``` python
@llm_prompt
def simulate_conversation(human_input:str, agent_role:str="a pirate"):
    """
    ## System message
     - note the `:system` suffix inside the <prompt:_role_> tag

    ```<prompt:system>
    You are a {agent_role} hacker. You mus act like one.
    You reply always in code, using python or javascript code block...
    for example:

    ... do not reply with anything else.. just with code - respecting your role.
    ```

    # human message
    (we are using the real role that are enforced by the LLM - GPT supports system, assistant, user)
    ``` <prompt:user>
    Helo, who are you
    ```
    a reply:

    ``` <prompt:assistant>
    \``` python <<- escaping inner code block with \ that should be part of the prompt
    def hello():
        print("Argh... hello you pesky pirate")
    \```
    ```

    we can also add some history using placeholder
    ```<prompt:placeholder>
    {history}
    ```
    ```<prompt:user>
    {human_input}
    ```

    Now only to code block above will be used as a prompt, and the rest of the docstring will be used as a description for developers.
    (It has also a nice benefit that IDE (like VS code) will display the prompt properly (not trying to parse it as markdown, and thus not showing new lines properly))
    """
    pass

```

the roles here are model native roles (assistant, user, system for chatGPT)


# LocalAI

>[LocalAI](https://localai.io/) is the free, Open Source OpenAI alternative.
> `LocalAI` act as a drop-in replacement REST API that’s compatible with OpenAI API
> specifications for local inferencing. It allows you to run LLMs, generate images,
> audio (and not only) locally or on-prem with consumer grade hardware,
> supporting multiple model families and architectures.

## Installation and Setup

We have to install several python packages:

```bash
pip install tenacity openai
```

## Embedding models

See a [usage example](/docs/integrations/text_embedding/localai).

```python
from langchain_community.embeddings import LocalAIEmbeddings
```


# GPT4All

This page covers how to use the `GPT4All` wrapper within LangChain. The tutorial is divided into two parts: installation and setup, followed by usage with an example.

## Installation and Setup

- Install the Python package with `pip install gpt4all`
- Download a [GPT4All model](https://gpt4all.io/index.html) and place it in your desired directory

In this example, we are using `mistral-7b-openorca.Q4_0.gguf`:

```bash
mkdir models
wget https://gpt4all.io/models/gguf/mistral-7b-openorca.Q4_0.gguf -O models/mistral-7b-openorca.Q4_0.gguf
```

## Usage

### GPT4All

To use the GPT4All wrapper, you need to provide the path to the pre-trained model file and the model's configuration.

```python
from langchain_community.llms import GPT4All

# Instantiate the model. Callbacks support token-wise streaming
model = GPT4All(model="./models/mistral-7b-openorca.Q4_0.gguf", n_threads=8)

# Generate text
response = model.invoke("Once upon a time, ")
```

You can also customize the generation parameters, such as `n_predict`, `temp`, `top_p`, `top_k`, and others.

To stream the model's predictions, add in a CallbackManager.

```python
from langchain_community.llms import GPT4All
from langchain.callbacks.streaming_stdout import StreamingStdOutCallbackHandler

# There are many CallbackHandlers supported, such as
# from langchain.callbacks.streamlit import StreamlitCallbackHandler

callbacks = [StreamingStdOutCallbackHandler()]
model = GPT4All(model="./models/mistral-7b-openorca.Q4_0.gguf", n_threads=8)

# Generate text. Tokens are streamed through the callback manager.
model.invoke("Once upon a time, ", callbacks=callbacks)
```

## Model File

You can download model files from the GPT4All client. You can download the client from the [GPT4All](https://gpt4all.io/index.html) website.

For a more detailed walkthrough of this, see [this notebook](/docs/integrations/llms/gpt4all)


# DocArray

> [DocArray](https://docarray.jina.ai/) is a library for nested, unstructured, multimodal data in transit,
> including text, image, audio, video, 3D mesh, etc. It allows deep-learning engineers to efficiently process,
> embed, search, recommend, store, and transfer multimodal data with a Pythonic API.

## Installation and Setup

We need to install `docarray` python package.

```bash
pip install docarray
```

## Vector Store

LangChain provides an access to the `In-memory` and `HNSW` vector stores from the `DocArray` library.

See a [usage example](/docs/integrations/vectorstores/docarray_hnsw).

```python
from langchain_community.vectorstores import DocArrayHnswSearch
```
See a [usage example](/docs/integrations/vectorstores/docarray_in_memory).

```python
from langchain_community.vectorstores DocArrayInMemorySearch
```

## Retriever

See a [usage example](/docs/integrations/retrievers/docarray_retriever).

```python
from langchain_community.retrievers import DocArrayRetriever
```


# Airbyte

>[Airbyte](https://github.com/airbytehq/airbyte) is a data integration platform for ELT pipelines from APIs,
> databases & files to warehouses & lakes. It has the largest catalog of ELT connectors to data warehouses and databases.

## Installation and Setup

```bash
pip install -U langchain-airbyte
```

:::note

Currently, the `langchain-airbyte` library does not support Pydantic v2.
Please downgrade to Pydantic v1 to use this package.

This package also currently requires Python 3.10+.

:::

The integration package doesn't require any global environment variables that need to be
set, but some integrations (e.g. `source-github`) may need credentials passed in.

## Document loader

### AirbyteLoader

See a [usage example](/docs/integrations/document_loaders/airbyte).

```python
from langchain_airbyte import AirbyteLoader
```


# LlamaIndex

>[LlamaIndex](https://www.llamaindex.ai/) is the leading data framework for building LLM applications

## Installation and Setup

You need to install the `llama-index` python package.

```bash
pip install llama-index
```

See the [installation instructions](https://docs.llamaindex.ai/en/stable/getting_started/installation/).

## Retrievers

### LlamaIndexRetriever

>It is used for the question-answering with sources over an LlamaIndex data structure.

```python
from langchain_community.retrievers.llama_index import LlamaIndexRetriever
```

### LlamaIndexGraphRetriever

>It is used for question-answering with sources over an LlamaIndex graph data structure.

```python
from langchain_community.retrievers.llama_index import LlamaIndexGraphRetriever
```


---
keywords: [pinecone]
---

# Pinecone

>[Pinecone](https://docs.pinecone.io/docs/overview) is a vector database with broad functionality.

## Installation and Setup

Install the Python SDK:

```bash
pip install langchain-pinecone
```

## Vector store

There exists a wrapper around Pinecone indexes, allowing you to use it as a vectorstore,
whether for semantic search or example selection.

```python
from langchain_pinecone import PineconeVectorStore
```

For a more detailed walkthrough of the Pinecone vectorstore, see [this notebook](/docs/integrations/vectorstores/pinecone)

## Retrievers

### Pinecone Hybrid Search

```bash
pip install pinecone-client pinecone-text
```

```python
from langchain_community.retrievers import (
    PineconeHybridSearchRetriever,
)
```

For more detailed information, see [this notebook](/docs/integrations/retrievers/pinecone_hybrid_search).

### Self Query retriever

Pinecone vector store can be used as a retriever for self-querying.

For more detailed information, see [this notebook](/docs/integrations/retrievers/self_query/pinecone).


# Unstructured

>The `unstructured` package from
[Unstructured.IO](https://www.unstructured.io/) extracts clean text from raw source documents like
PDFs and Word documents.
This page covers how to use the [`unstructured`](https://github.com/Unstructured-IO/unstructured)
ecosystem within LangChain.

## Installation and Setup

If you are using a loader that runs locally, use the following steps to get `unstructured` and its
dependencies running.

- For the smallest installation footprint and to take advantage of features not available in the
  open-source `unstructured` package, install the Python SDK with `pip install unstructured-client`
  along with `pip install langchain-unstructured` to use the `UnstructuredLoader` and partition
  remotely against the Unstructured API. This loader lives
  in a LangChain partner repo instead of the `langchain-community` repo and you will need an
  `api_key`, which you can generate a free key [here](https://unstructured.io/api-key/).
    - Unstructured's documentation for the sdk can be found here:
      https://docs.unstructured.io/api-reference/api-services/sdk

- To run everything locally, install the open-source python package with `pip install unstructured`
  along with `pip install langchain-community` and use the same `UnstructuredLoader` as mentioned above.
    - You can install document specific dependencies with extras, e.g. `pip install "unstructured[docx]"`.
    - To install the dependencies for all document types, use `pip install "unstructured[all-docs]"`.
- Install the following system dependencies if they are not already available on your system with e.g. `brew install` for Mac.
  Depending on what document types you're parsing, you may not need all of these.
    - `libmagic-dev` (filetype detection)
    - `poppler-utils` (images and PDFs)
    - `tesseract-ocr`(images and PDFs)
    - `qpdf` (PDFs)
    - `libreoffice` (MS Office docs)
    - `pandoc` (EPUBs)
- When running locally, Unstructured also recommends using Docker [by following this
  guide](https://docs.unstructured.io/open-source/installation/docker-installation) to ensure all
  system dependencies are installed correctly.

The Unstructured API requires API keys to make requests.
You can request an API key [here](https://unstructured.io/api-key-hosted) and start using it today!
Checkout the README [here](https://github.com/Unstructured-IO/unstructured-api) here to get started making API calls.
We'd love to hear your feedback, let us know how it goes in our [community slack](https://join.slack.com/t/unstructuredw-kbe4326/shared_invite/zt-1x7cgo0pg-PTptXWylzPQF9xZolzCnwQ).
And stay tuned for improvements to both quality and performance!
Check out the instructions
[here](https://github.com/Unstructured-IO/unstructured-api#dizzy-instructions-for-using-the-docker-image) if you'd like to self-host the Unstructured API or run it locally.

## Data Loaders

The primary usage of `Unstructured` is in data loaders.

### UnstructuredLoader

See a [usage example](/docs/integrations/document_loaders/unstructured_file) to see how you can use
this loader for both partitioning locally and remotely with the serverless Unstructured API.

```python
from langchain_unstructured import UnstructuredLoader
```

### UnstructuredCHMLoader

`CHM` means `Microsoft Compiled HTML Help`.

```python
from langchain_community.document_loaders import UnstructuredCHMLoader
```

### UnstructuredCSVLoader

A `comma-separated values` (`CSV`) file is a delimited text file that uses
a comma to separate values. Each line of the file is a data record.
Each record consists of one or more fields, separated by commas.

See a [usage example](/docs/integrations/document_loaders/csv#unstructuredcsvloader).

```python
from langchain_community.document_loaders import UnstructuredCSVLoader
```

### UnstructuredEmailLoader

See a [usage example](/docs/integrations/document_loaders/email).

```python
from langchain_community.document_loaders import UnstructuredEmailLoader
```

### UnstructuredEPubLoader

[EPUB](https://en.wikipedia.org/wiki/EPUB) is an `e-book file format` that uses
the “.epub” file extension. The term is short for electronic publication and
is sometimes styled `ePub`. `EPUB` is supported by many e-readers, and compatible
software is available for most smartphones, tablets, and computers.

See a [usage example](/docs/integrations/document_loaders/epub).

```python
from langchain_community.document_loaders import UnstructuredEPubLoader
```

### UnstructuredExcelLoader

See a [usage example](/docs/integrations/document_loaders/microsoft_excel).

```python
from langchain_community.document_loaders import UnstructuredExcelLoader
```

### UnstructuredFileIOLoader

See a [usage example](/docs/integrations/document_loaders/google_drive#passing-in-optional-file-loaders).

```python
from langchain_community.document_loaders import UnstructuredFileIOLoader
```

### UnstructuredHTMLLoader

See a [usage example](/docs/how_to/document_loader_html).

```python
from langchain_community.document_loaders import UnstructuredHTMLLoader
```

### UnstructuredImageLoader

See a [usage example](/docs/integrations/document_loaders/image).

```python
from langchain_community.document_loaders import UnstructuredImageLoader
```

### UnstructuredMarkdownLoader

See a [usage example](/docs/integrations/vectorstores/starrocks).

```python
from langchain_community.document_loaders import UnstructuredMarkdownLoader
```

### UnstructuredODTLoader

The `Open Document Format for Office Applications (ODF)`, also known as `OpenDocument`,
is an open file format for word processing documents, spreadsheets, presentations
and graphics and using ZIP-compressed XML files. It was developed with the aim of
providing an open, XML-based file format specification for office applications.

See a [usage example](/docs/integrations/document_loaders/odt).

```python
from langchain_community.document_loaders import UnstructuredODTLoader
```

### UnstructuredOrgModeLoader

An [Org Mode](https://en.wikipedia.org/wiki/Org-mode) document is a document editing, formatting, and organizing mode, designed for notes, planning, and authoring within the free software text editor Emacs.

See a [usage example](/docs/integrations/document_loaders/org_mode).

```python
from langchain_community.document_loaders import UnstructuredOrgModeLoader
```

### UnstructuredPDFLoader

See a [usage example](/docs/how_to/document_loader_pdf#using-unstructured).

```python
from langchain_community.document_loaders import UnstructuredPDFLoader
```

### UnstructuredPowerPointLoader

See a [usage example](/docs/integrations/document_loaders/microsoft_powerpoint).

```python
from langchain_community.document_loaders import UnstructuredPowerPointLoader
```

### UnstructuredRSTLoader

A `reStructured Text` (`RST`) file is a file format for textual data
used primarily in the Python programming language community for technical documentation.

See a [usage example](/docs/integrations/document_loaders/rst).

```python
from langchain_community.document_loaders import UnstructuredRSTLoader
```

### UnstructuredRTFLoader

See a usage example in the API documentation.

```python
from langchain_community.document_loaders import UnstructuredRTFLoader
```

### UnstructuredTSVLoader

A `tab-separated values` (`TSV`) file is a simple, text-based file format for storing tabular data.
Records are separated by newlines, and values within a record are separated by tab characters.

See a [usage example](/docs/integrations/document_loaders/tsv).

```python
from langchain_community.document_loaders import UnstructuredTSVLoader
```

### UnstructuredURLLoader

See a [usage example](/docs/integrations/document_loaders/url).

```python
from langchain_community.document_loaders import UnstructuredURLLoader
```

### UnstructuredWordDocumentLoader

See a [usage example](/docs/integrations/document_loaders/microsoft_word#using-unstructured).

```python
from langchain_community.document_loaders import UnstructuredWordDocumentLoader
```

### UnstructuredXMLLoader

See a [usage example](/docs/integrations/document_loaders/xml).

```python
from langchain_community.document_loaders import UnstructuredXMLLoader
```


# Astra DB

> [DataStax Astra DB](https://docs.datastax.com/en/astra/home/astra.html) is a serverless
> vector-capable database built on `Apache Cassandra®`and made conveniently available
> through an easy-to-use JSON API.

See a [tutorial provided by DataStax](https://docs.datastax.com/en/astra/astra-db-vector/tutorials/chatbot.html).

## Installation and Setup

Install the following Python package:
```bash
pip install "langchain-astradb>=0.1.0"
```

Get the [connection secrets](https://docs.datastax.com/en/astra/astra-db-vector/get-started/quickstart.html).
Set up the following environment variables:

```python
ASTRA_DB_APPLICATION_TOKEN="TOKEN"
ASTRA_DB_API_ENDPOINT="API_ENDPOINT"
```

## Vector Store

```python
from langchain_astradb import AstraDBVectorStore

vector_store = AstraDBVectorStore(
    embedding=my_embedding,
    collection_name="my_store",
    api_endpoint=ASTRA_DB_API_ENDPOINT,
    token=ASTRA_DB_APPLICATION_TOKEN,
)
```

Learn more in the [example notebook](/docs/integrations/vectorstores/astradb).

See the [example provided by DataStax](https://docs.datastax.com/en/astra/astra-db-vector/integrations/langchain.html).

## Chat message history

```python
from langchain_astradb import AstraDBChatMessageHistory

message_history = AstraDBChatMessageHistory(
    session_id="test-session",
    api_endpoint=ASTRA_DB_API_ENDPOINT,
    token=ASTRA_DB_APPLICATION_TOKEN,
)
```

See the [usage example](/docs/integrations/memory/astradb_chat_message_history#example).

## LLM Cache

```python
from langchain.globals import set_llm_cache
from langchain_astradb import AstraDBCache

set_llm_cache(AstraDBCache(
    api_endpoint=ASTRA_DB_API_ENDPOINT,
    token=ASTRA_DB_APPLICATION_TOKEN,
))
```

Learn more in the [example notebook](/docs/integrations/llm_caching#astra-db-caches) (scroll to the Astra DB section).

## Semantic LLM Cache

```python
from langchain.globals import set_llm_cache
from langchain_astradb import AstraDBSemanticCache

set_llm_cache(AstraDBSemanticCache(
    embedding=my_embedding,
    api_endpoint=ASTRA_DB_API_ENDPOINT,
    token=ASTRA_DB_APPLICATION_TOKEN,
))
```

Learn more in the [example notebook](/docs/integrations/llm_caching#astra-db-caches) (scroll to the appropriate section).

Learn more in the [example notebook](/docs/integrations/memory/astradb_chat_message_history).

## Document loader

```python
from langchain_astradb import AstraDBLoader

loader = AstraDBLoader(
    collection_name="my_collection",
    api_endpoint=ASTRA_DB_API_ENDPOINT,
    token=ASTRA_DB_APPLICATION_TOKEN,
)
```

Learn more in the [example notebook](/docs/integrations/document_loaders/astradb).

## Self-querying retriever

```python
from langchain_astradb import AstraDBVectorStore
from langchain.retrievers.self_query.base import SelfQueryRetriever

vector_store = AstraDBVectorStore(
    embedding=my_embedding,
    collection_name="my_store",
    api_endpoint=ASTRA_DB_API_ENDPOINT,
    token=ASTRA_DB_APPLICATION_TOKEN,
)

retriever = SelfQueryRetriever.from_llm(
    my_llm,
    vector_store,
    document_content_description,
    metadata_field_info
)
```

Learn more in the [example notebook](/docs/integrations/retrievers/self_query/astradb).

## Store

```python
from langchain_astradb import AstraDBStore

store = AstraDBStore(
    collection_name="my_kv_store",
    api_endpoint=ASTRA_DB_API_ENDPOINT,
    token=ASTRA_DB_APPLICATION_TOKEN,
)
```

Learn more in the [example notebook](/docs/integrations/stores/astradb#astradbstore).

## Byte Store

```python
from langchain_astradb import AstraDBByteStore

store = AstraDBByteStore(
    collection_name="my_kv_store",
    api_endpoint=ASTRA_DB_API_ENDPOINT,
    token=ASTRA_DB_APPLICATION_TOKEN,
)
```

Learn more in the [example notebook](/docs/integrations/stores/astradb#astradbbytestore).


# Motherduck

>[Motherduck](https://motherduck.com/) is a managed DuckDB-in-the-cloud service.

## Installation and Setup

First, you need to install `duckdb` python package.

```bash
pip install duckdb
```

You will also need to sign up for an account at [Motherduck](https://motherduck.com/)

After that, you should set up a connection string - we mostly integrate with Motherduck through SQLAlchemy.
The connection string is likely in the form:

```
token="..."

conn_str = f"duckdb:///md:{token}@my_db"
```

## SQLChain

You can use the SQLChain to query data in your Motherduck instance in natural language.

```
from langchain_openai import OpenAI
from langchain_community.utilities import SQLDatabase
from langchain_experimental.sql import SQLDatabaseChain
db = SQLDatabase.from_uri(conn_str)
db_chain = SQLDatabaseChain.from_llm(OpenAI(temperature=0), db, verbose=True)
```

From here, see the [SQL Chain](/docs/how_to#qa-over-sql--csv) documentation on how to use.

## LLMCache

You can also easily use Motherduck to cache LLM requests.
Once again this is done through the SQLAlchemy wrapper.

```
import sqlalchemy
from langchain.globals import set_llm_cache
eng = sqlalchemy.create_engine(conn_str)
set_llm_cache(SQLAlchemyCache(engine=eng))
```

From here, see the [LLM Caching](/docs/integrations/llm_caching) documentation on how to use.


# Llama.cpp

>[llama.cpp python](https://github.com/abetlen/llama-cpp-python) library is a simple Python bindings for `@ggerganov`
>[llama.cpp](https://github.com/ggerganov/llama.cpp).
>
>This package provides:
>
> - Low-level access to C API via ctypes interface.
> - High-level Python API for text completion
>   - `OpenAI`-like API
>   - `LangChain` compatibility
>   - `LlamaIndex` compatibility
> - OpenAI compatible web server
>   - Local Copilot replacement
>   - Function Calling support
>   - Vision API support
>   - Multiple Models

## Installation and Setup

- Install the Python package
  ```bash
  pip install llama-cpp-python
  ````
- Download one of the [supported models](https://github.com/ggerganov/llama.cpp#description) and convert them to the llama.cpp format per the [instructions](https://github.com/ggerganov/llama.cpp)

## Chat models

See a [usage example](/docs/integrations/chat/llamacpp).

```python
from langchain_community.chat_models import ChatLlamaCpp
```

## LLMs

See a [usage example](/docs/integrations/llms/llamacpp).

```python
from langchain_community.llms import LlamaCpp
```

## Embedding models

See a [usage example](/docs/integrations/text_embedding/llamacpp).

```python
from langchain_community.embeddings import LlamaCppEmbeddings
```


# Google

All functionality related to [Google Cloud Platform](https://cloud.google.com/) and other `Google` products.

## Chat models

We recommend individual developers to start with Gemini API (`langchain-google-genai`) and move to Vertex AI (`langchain-google-vertexai`) when they need access to commercial support and higher rate limits. If you’re already Cloud-friendly or Cloud-native, then you can get started in Vertex AI straight away.
Please see [here](https://ai.google.dev/gemini-api/docs/migrate-to-cloud) for more information.

### Google Generative AI

Access GoogleAI `Gemini` models such as `gemini-pro` and `gemini-pro-vision` through the `ChatGoogleGenerativeAI` class.

```bash
pip install -U langchain-google-genai
```

Configure your API key.

```bash
export GOOGLE_API_KEY=your-api-key
```

```python
from langchain_google_genai import ChatGoogleGenerativeAI

llm = ChatGoogleGenerativeAI(model="gemini-pro")
llm.invoke("Sing a ballad of LangChain.")
```

Gemini vision model supports image inputs when providing a single chat message.

```python
from langchain_core.messages import HumanMessage
from langchain_google_genai import ChatGoogleGenerativeAI

llm = ChatGoogleGenerativeAI(model="gemini-pro-vision")

message = HumanMessage(
    content=[
        {
            "type": "text",
            "text": "What's in this image?",
        },  # You can optionally provide text parts
        {"type": "image_url", "image_url": "https://picsum.photos/seed/picsum/200/300"},
    ]
)
llm.invoke([message])
```

The value of image_url can be any of the following:

- A public image URL
- A gcs file (e.g., "gcs://path/to/file.png")
- A local file path
- A base64 encoded image (e.g., data:image/png;base64,abcd124)
- A PIL image

### Vertex AI

Access chat models like `Gemini` via Google Cloud.

We need to install `langchain-google-vertexai` python package.

```bash
pip install langchain-google-vertexai
```

See a [usage example](/docs/integrations/chat/google_vertex_ai_palm).

```python
from langchain_google_vertexai import ChatVertexAI
```

### Anthropic on Vertex AI Model Garden

See a [usage example](/docs/integrations/llms/google_vertex_ai_palm).

```python
from langchain_google_vertexai.model_garden import ChatAnthropicVertex
```

### Llama on Vertex AI Model Garden

```python
from langchain_google_vertexai.model_garden_maas.llama import VertexModelGardenLlama
```

### Mistral on Vertex AI Model Garden

```python
from langchain_google_vertexai.model_garden_maas.mistral import VertexModelGardenMistral
```

### Gemma local from Hugging Face

>Local `Gemma` model loaded from `HuggingFace`.

We need to install `langchain-google-vertexai` python package.

```bash
pip install langchain-google-vertexai
```

```python
from langchain_google_vertexai.gemma import GemmaChatLocalHF
```

### Gemma local from Kaggle

>Local `Gemma` model loaded from `Kaggle`.

We need to install `langchain-google-vertexai` python package.

```bash
pip install langchain-google-vertexai
```

```python
from langchain_google_vertexai.gemma import GemmaChatLocalKaggle
```

### Gemma on Vertex AI Model Garden

We need to install `langchain-google-vertexai` python package.

```bash
pip install langchain-google-vertexai
```

```python
from langchain_google_vertexai.gemma import GemmaChatVertexAIModelGarden
```

### Vertex AI image captioning

>Implementation of the `Image Captioning model` as a chat.

We need to install `langchain-google-vertexai` python package.

```bash
pip install langchain-google-vertexai
```

```python
from langchain_google_vertexai.vision_models import VertexAIImageCaptioningChat
```

### Vertex AI image editor

>Given an image and a prompt, edit the image. Currently only supports mask-free editing.

We need to install `langchain-google-vertexai` python package.

```bash
pip install langchain-google-vertexai
```

```python
from langchain_google_vertexai.vision_models import VertexAIImageEditorChat
```

### Vertex AI image generator

>Generates an image from a prompt.

We need to install `langchain-google-vertexai` python package.

```bash
pip install langchain-google-vertexai
```

```python
from langchain_google_vertexai.vision_models import VertexAIImageGeneratorChat
```

### Vertex AI visual QnA

>Chat implementation of a visual QnA model

We need to install `langchain-google-vertexai` python package.

```bash
pip install langchain-google-vertexai
```

```python
from langchain_google_vertexai.vision_models import VertexAIVisualQnAChat
```

## LLMs

### Google Generative AI

Access GoogleAI `Gemini` models such as `gemini-pro` and `gemini-pro-vision` through the `GoogleGenerativeAI` class.

Install python package.

```bash
pip install langchain-google-genai
```

See a [usage example](/docs/integrations/llms/google_ai).

```python
from langchain_google_genai import GoogleGenerativeAI
```

### Vertex AI Model Garden

Access `PaLM` and hundreds of OSS models via `Vertex AI Model Garden` service.

We need to install `langchain-google-vertexai` python package.

```bash
pip install langchain-google-vertexai
```

See a [usage example](/docs/integrations/llms/google_vertex_ai_palm#vertex-model-garden).

```python
from langchain_google_vertexai import VertexAIModelGarden
```

### Gemma local from Hugging Face

>Local `Gemma` model loaded from `HuggingFace`.

We need to install `langchain-google-vertexai` python package.

```bash
pip install langchain-google-vertexai
```

```python
from langchain_google_vertexai.gemma import GemmaLocalHF
```

### Gemma local from Kaggle

>Local `Gemma` model loaded from `Kaggle`.

We need to install `langchain-google-vertexai` python package.

```bash
pip install langchain-google-vertexai
```

```python
from langchain_google_vertexai.gemma import GemmaLocalKaggle
```

### Gemma on Vertex AI Model Garden

We need to install `langchain-google-vertexai` python package.

```bash
pip install langchain-google-vertexai
```

```python
from langchain_google_vertexai.gemma import GemmaVertexAIModelGarden
```

### Vertex AI image captioning

>Implementation of the `Image Captioning model` as an LLM.

We need to install `langchain-google-vertexai` python package.

```bash
pip install langchain-google-vertexai
```

```python
from langchain_google_vertexai.vision_models import VertexAIImageCaptioning
```

## Embedding models

### Google Generative AI embedding

See a [usage example](/docs/integrations/text_embedding/google_generative_ai).

```bash
pip install -U langchain-google-genai
```

Configure your API key.

```bash
export GOOGLE_API_KEY=your-api-key
```

```python
from langchain_google_genai import GoogleGenerativeAIEmbeddings
```

### Google Generative AI server-side embedding

Install the python package:

```bash
pip install langchain-google-genai
```

```python
from langchain_google_genai.google_vector_store import ServerSideEmbedding
```

### Vertex AI

We need to install `langchain-google-vertexai` python package.

```bash
pip install langchain-google-vertexai
```

See a [usage example](/docs/integrations/text_embedding/google_vertex_ai_palm).

```python
from langchain_google_vertexai import VertexAIEmbeddings
```

### Palm embedding

We need to install `langchain-community` python package.

```bash
pip install langchain-community
```

```python
from langchain_community.embeddings.google_palm import GooglePalmEmbeddings
```

## Document Loaders

### AlloyDB for PostgreSQL

> [Google Cloud AlloyDB](https://cloud.google.com/alloydb) is a fully managed relational database service that offers high performance, seamless integration, and impressive scalability on Google Cloud. AlloyDB is 100% compatible with PostgreSQL.

Install the python package:

```bash
pip install langchain-google-alloydb-pg
```

See [usage example](/docs/integrations/document_loaders/google_alloydb).

```python
from langchain_google_alloydb_pg import AlloyDBEngine, AlloyDBLoader
```

### BigQuery

> [Google Cloud BigQuery](https://cloud.google.com/bigquery) is a serverless and cost-effective enterprise data warehouse that works across clouds and scales with your data in Google Cloud.

We need to install `langchain-google-community` with Big Query dependencies:

```bash
pip install langchain-google-community[bigquery]
```


Upstash offers developers serverless databases and messaging
platforms to build powerful applications without having to worry
about the operational complexity of running databases at scale.

One significant advantage of Upstash is that their databases support HTTP and all of their SDKs use HTTP.
This means that you can run this in serverless platforms, edge or any platform that does not support TCP connections.

Currently, there are two Upstash integrations available for LangChain:
Upstash Vector as a vector embedding database and Upstash Redis as a cache and memory store.

# Upstash Vector

Upstash Vector is a serverless vector database that can be used to store and query vectors.

## Installation

Create a new serverless vector database at the [Upstash Console](https://console.upstash.com/vector).
Select your preferred distance metric and dimension count according to your model.

Install the Upstash Vector Python SDK with `pip install upstash-vector`.
The Upstash Vector integration in langchain is a wrapper for the Upstash Vector Python SDK. That's why the `upstash-vector` package is required.

## Integrations

Create a `UpstashVectorStore` object using credentials from the Upstash Console.
You also need to pass in an `Embeddings` object which can turn text into vector embeddings.

```python
from langchain_community.vectorstores.upstash import UpstashVectorStore
import os

os.environ["UPSTASH_VECTOR_REST_URL"] = "<UPSTASH_VECTOR_REST_URL>"
os.environ["UPSTASH_VECTOR_REST_TOKEN"] = "<UPSTASH_VECTOR_REST_TOKEN>"

store = UpstashVectorStore(
    embedding=embeddings
)
```

An alternative way of `UpstashVectorStore` is to pass `embedding=True`. This is a unique
feature of the `UpstashVectorStore` thanks to the ability of the Upstash Vector indexes
to have an associated embedding model. In this configuration, documents we want to insert or
queries we want to search for are simply sent to Upstash Vector as text. In the background,
Upstash Vector embeds these text and executes the request with these embeddings. To use this
feature, [create an Upstash Vector index by selecting a model](https://upstash.com/docs/vector/features/embeddingmodels#using-a-model)
and simply pass `embedding=True`:

```python
from langchain_community.vectorstores.upstash import UpstashVectorStore
import os

os.environ["UPSTASH_VECTOR_REST_URL"] = "<UPSTASH_VECTOR_REST_URL>"
os.environ["UPSTASH_VECTOR_REST_TOKEN"] = "<UPSTASH_VECTOR_REST_TOKEN>"

store = UpstashVectorStore(
    embedding=True
)
```

See [Upstash Vector documentation](https://upstash.com/docs/vector/features/embeddingmodels)
for more detail on embedding models.

## Namespaces
You can use namespaces to partition your data in the index. Namespaces are useful when you want to query over huge amount of data, and you want to partition the data to make the queries faster. When you use namespaces, there won't be post-filtering on the results which will make the query results more precise.

```python
from langchain_community.vectorstores.upstash import UpstashVectorStore
import os

os.environ["UPSTASH_VECTOR_REST_URL"] = "<UPSTASH_VECTOR_REST_URL>"
os.environ["UPSTASH_VECTOR_REST_TOKEN"] = "<UPSTASH_VECTOR_REST_TOKEN>"

store = UpstashVectorStore(
    embedding=embeddings
    namespace="my_namespace"
)
```

### Inserting Vectors

```python
from langchain.text_splitter import CharacterTextSplitter
from langchain_community.document_loaders import TextLoader
from langchain_openai import OpenAIEmbeddings

loader = TextLoader("../../modules/state_of_the_union.txt")
documents = loader.load()
text_splitter = CharacterTextSplitter(chunk_size=1000, chunk_overlap=0)
docs = text_splitter.split_documents(documents)

# Create a new embeddings object
embeddings = OpenAIEmbeddings()

# Create a new UpstashVectorStore object
store = UpstashVectorStore(
    embedding=embeddings
)

# Insert the document embeddings into the store
store.add_documents(docs)
```

When inserting documents, first they are embedded using the `Embeddings` object.

Most embedding models can embed multiple documents at once, so the documents are batched and embedded in parallel.
The size of the batch can be controlled using the `embedding_chunk_size` parameter.

The embedded vectors are then stored in the Upstash Vector database. When they are sent, multiple vectors are batched together to reduce the number of HTTP requests.
The size of the batch can be controlled using the `batch_size` parameter. Upstash Vector has a limit of 1000 vectors per batch in the free tier.

```python
store.add_documents(
    documents,
    batch_size=100,
    embedding_chunk_size=200
)
```

### Querying Vectors

Vectors can be queried using a text query or another vector.

The returned value is a list of Document objects.

```python
result = store.similarity_search(
    "The United States of America",
    k=5
)
```

Or using a vector:

```python
vector = embeddings.embed_query("Hello world")

result = store.similarity_search_by_vector(
    vector,
    k=5
)
```

When searching, you can also utilize the `filter` parameter which will allow you to filter by metadata:

```python
result = store.similarity_search(
    "The United States of America",
    k=5,
    filter="type = 'country'"
)
```

See [Upstash Vector documentation](https://upstash.com/docs/vector/features/filtering)
for more details on metadata filtering.

### Deleting Vectors

Vectors can be deleted by their IDs.

```python
store.delete(["id1", "id2"])
```

### Getting information about the store

You can get information about your database like the distance metric dimension using the info function.

When an insert happens, the database an indexing takes place. While this is happening new vectors can not be queried. `pendingVectorCount` represents the number of vector that are currently being indexed.

```python
info = store.info()
print(info)

# Output:
# {'vectorCount': 44, 'pendingVectorCount': 0, 'indexSize': 2642412, 'dimension': 1536, 'similarityFunction': 'COSINE'}
```

# Upstash Redis

This page covers how to use [Upstash Redis](https://upstash.com/redis) with LangChain.

## Installation and Setup
- Upstash Redis Python SDK can be installed with `pip install upstash-redis`
- A globally distributed, low-latency and highly available database can be created at the [Upstash Console](https://console.upstash.com)

## Integrations
All of Upstash-LangChain integrations are based on `upstash-redis` Python SDK being utilized as wrappers for LangChain.
This SDK utilizes Upstash Redis DB by giving UPSTASH_REDIS_REST_URL and UPSTASH_REDIS_REST_TOKEN parameters from the console.

### Cache

[Upstash Redis](https://upstash.com/redis) can be used as a cache for LLM prompts and responses.

To import this cache:
```python
from langchain.cache import UpstashRedisCache
```

To use with your LLMs:
```python
import langchain
from upstash_redis import Redis

URL = "<UPSTASH_REDIS_REST_URL>"
TOKEN = "<UPSTASH_REDIS_REST_TOKEN>"

langchain.llm_cache = UpstashRedisCache(redis_=Redis(url=URL, token=TOKEN))
```

### Memory

See a [usage example](/docs/integrations/memory/upstash_redis_chat_message_history).

```python
from langchain_community.chat_message_histories import (
    UpstashRedisChatMessageHistory,
)
```