**Supports Streaming**: Whether the output parser supports streaming.

**Input Type**: Expected input type. Most output parsers work on both strings and messages, but some (like OpenAI Functions) need a message with specific arguments.

**Output Type**: The output type of the object returned by the parser.

**Description**: Our commentary on this output parser and when to use it.

| Name                                                                                | Supports Streaming | Input Type                | Output Type                       | Description                                                                                |
| ------------------------------------------------------------------------------------------------------------- | ------------------ | ------------------------- | --------------------------------- | --------------------------------------------------------------------------------------------------------------------------------------------------------- |
| [JSON](https://api.js.langchain.com/classes/langchain_core.output_parsers.JsonOutputParser.html)              | ✅                 | `string` \| `BaseMessage` | `Promise<T>`                      | Returns a JSON object as specified. You can specify a Zod schema and it will return JSON for that model.                                                  |
| [XML](https://api.js.langchain.com/classes/langchain_core.output_parsers.XMLOutputParser.html)                | ✅                 | `string` \| `BaseMessage` | `Promise<XMLResult>`              | Returns a object of tags. Use when XML output is needed. Use with models that are good at writing XML (like Anthropic's).                                 |
| [CSV](https://api.js.langchain.com/classes/langchain_core.output_parsers.CommaSeparatedListOutputParser.html) | ✅                 | `string` \| `BaseMessage` | `Array[string]`                   | Returns an array of comma separated values.                                                                                |
| [Structured](https://api.js.langchain.com/classes/langchain_core.output_parsers.StructuredOutputParser.html)  |                    | `string` \| `BaseMessage` | `Promise<TypeOf<T>>`              | Parse structured JSON from an LLM response.                                                                                |
| [HTTP](https://api.js.langchain.com/classes/langchain.output_parsers.HttpResponseOutputParser.html)           | ✅                 | `string`                  | `Promise<Uint8Array>`             | Parse an LLM response to then send over HTTP(s). Useful when invoking the LLM on the server/edge, and then sending the content/stream back to the client. |
| [Bytes](https://api.js.langchain.com/classes/langchain_core.output_parsers.BytesOutputParser.html)            | ✅                 | `string` \| `BaseMessage` | `Promise<Uint8Array>`             | Parse an LLM response to then send over HTTP(s). Useful for streaming LLM responses from the server/edge to the client.                                   |
| [Datetime](https://api.js.langchain.com/classes/langchain.output_parsers.DatetimeOutputParser.html)           |                    | `string`                  | `Promise<Date>`                   | Parses response into a `Date`.                                                                                |
| [Regex](https://api.js.langchain.com/classes/langchain.output_parsers.RegexParser.html)                       |                    | `string`                  | `Promise<Record<string, string>>` | Parses the given text using the regex pattern and returns a object with the parsed output.                                                                |

For specifics on how to use output parsers, see the [relevant how-to guides here](/docs/how_to/#output-parsers).

### Chat History

Most LLM applications have a conversational interface.
An essential component of a conversation is being able to refer to information introduced earlier in the conversation.
At bare minimum, a conversational system should be able to access some window of past messages directly.

The concept of `ChatHistory` refers to a class in LangChain which can be used to wrap an arbitrary chain.
This `ChatHistory` will keep track of inputs and outputs of the underlying chain, and append them as messages to a message database.
Future interactions will then load those messages and pass them into the chain as part of the input.

### Document

A Document object in LangChain contains information about some data. It has two attributes:

- `pageContent: string`: The content of this document. Currently is only a string.
- `metadata: Record<string, any>`: Arbitrary metadata associated with this document. Can track the document id, file name, etc.

### Document loaders

These classes load Document objects. LangChain has hundreds of integrations with various data sources to load data from: Slack, Notion, Google Drive, etc.

Each DocumentLoader has its own specific parameters, but they can all be invoked in the same way with the `.load` method.
An example use case is as follows:

```typescript
import { CSVLoader } from "@langchain/community/document_loaders/fs/csv";

const loader = new CSVLoader();
// <-- Integration specific parameters here

const docs = await loader.load();
```

For specifics on how to use document loaders, see the [relevant how-to guides here](/docs/how_to/#document-loaders).

### Text splitters

Once you've loaded documents, you'll often want to transform them to better suit your application. The simplest example is you may want to split a long document into smaller chunks that can fit into your model's context window. LangChain has a number of built-in document transformers that make it easy to split, combine, filter, and otherwise manipulate documents.

When you want to deal with long pieces of text, it is necessary to split up that text into chunks. As simple as this sounds, there is a lot of potential complexity here. Ideally, you want to keep the semantically related pieces of text together. What "semantically related" means could depend on the type of text. This notebook showcases several ways to do that.

At a high level, text splitters work as following:

1. Split the text up into small, semantically meaningful chunks (often sentences).
2. Start combining these small chunks into a larger chunk until you reach a certain size (as measured by some function).
3. Once you reach that size, make that chunk its own piece of text and then start creating a new chunk of text with some overlap (to keep context between chunks).

That means there are two different axes along which you can customize your text splitter:

1. How the text is split
2. How the chunk size is measured

For specifics on how to use text splitters, see the [relevant how-to guides here](/docs/how_to/#text-splitters).

### Embedding models

Embedding models create a vector representation of a piece of text. You can think of a vector as an array of numbers that captures the semantic meaning of the text.
By representing the text in this way, you can perform mathematical operations that allow you to do things like search for other pieces of text that are most similar in meaning.
These natural language search capabilities underpin many types of [context retrieval](/docs/concepts/#retrieval),
where we provide an LLM with the relevant data it needs to effectively respond to a query.

![](/img/embeddings.png)

The `Embeddings` class is a class designed for interfacing with text embedding models. There are many different embedding model providers (OpenAI, Cohere, Hugging Face, etc) and local models, and this class is designed to provide a standard interface for all of them.

The base Embeddings class in LangChain provides two methods: one for embedding documents and one for embedding a query. The former takes as input multiple texts, while the latter takes a single text. The reason for having these as two separate methods is that some embedding providers have different embedding methods for documents (to be searched over) vs queries (the search query itself).

For specifics on how to use embedding models, see the [relevant how-to guides here](/docs/how_to/#embedding-models).

### Vector stores {#vectorstore}

One of the most common ways to store and search over unstructured data is to embed it and store the resulting embedding vectors,
and then at query time to embed the unstructured query and retrieve the embedding vectors that are 'most similar' to the embedded query.
A vector store takes care of storing embedded data and performing vector search for you.

Most vector stores can also store metadata about embedded vectors and support filtering on that metadata before
similarity search, allowing you more control over returned documents.

Vectorstores can be converted to the retriever interface by doing:

```typescript
const vectorstore = new MyVectorStore();
const retriever = vectorstore.asRetriever();
```

For specifics on how to use vector stores, see the [relevant how-to guides here](/docs/how_to/#vectorstores).

### Retrievers

A retriever is an interface that returns relevant documents given an unstructured query.
They are more general than a vector store.
A retriever does not need to be able to store documents, only to return (or retrieve) them.
Retrievers can be created from vector stores, but are also broad enough to include [Exa search](/docs/integrations/retrievers/exa/) (web search) and [Amazon Kendra](/docs/integrations/retrievers/kendra-retriever/).

Retrievers accept a string query as input and return an array of `Document`s as output.

For specifics on how to use retrievers, see the [relevant how-to guides here](/docs/how_to/#retrievers).

### Key-value stores

For some techniques, such as [indexing and retrieval with multiple vectors per document](/docs/how_to/multi_vector/), having some sort of key-value (KV) storage is helpful.


LangChain includes a [`BaseStore`](https://api.js.langchain.com/classes/langchain_core.stores.BaseStore.html) interface,
which allows for storage of arbitrary data. However, LangChain components that require KV-storage accept a
more specific `BaseStore<string, Uint8Array>` instance that stores binary data (referred to as a `ByteStore`), and internally take care of
encoding and decoding data for their specific needs.

This means that as a user, you only need to think about one type of store rather than different ones for different types of data.

#### Interface

All [`BaseStores`](https://api.js.langchain.com/classes/langchain_core.stores.BaseStore.html) support the following interface. Note that the interface allows
for modifying **multiple** key-value pairs at once:

- `mget(keys: string[]): Promise<(undefined | Uint8Array)[]>`: get the contents of multiple keys, returning `None` if the key does not exist
- `mset(keyValuePairs: [string, Uint8Array][]): Promise<void>`: set the contents of multiple keys
- `mdelete(keys: string[]): Promise<void>`: delete multiple keys
- `yieldKeys(prefix?: string): AsyncGenerator<string>`: yield all keys in the store, optionally filtering by a prefix

For key-value store implementations, see [this section](/docs/integrations/stores/).

### Tools

Tools are utilities designed to be called by a model: their inputs are designed to be generated by models, and their outputs are designed to be passed back to models.
Tools are needed whenever you want a model to control parts of your code or call out to external APIs.

A tool consists of:

1. The name of the tool.
2. A description of what the tool does.
3. A JSON schema defining the inputs to the tool.
4. A function.

When a tool is bound to a model, the name, description and JSON schema are provided as context to the model.

Given a list of tools and a set of instructions, a model can request to call one or more tools with specific inputs.
Typical usage may look like the following:

```ts
// Define a list of tools
const tools = [...];
const llmWithTools = llm.bindTools([tool]);

const aiMessage = await llmWithTools.invoke("do xyz...");
// AIMessage(tool_calls=[ToolCall(...), ...], ...)
```

The `AIMessage` returned from the model MAY have `tool_calls` associated with it.
Read [this guide](/docs/concepts/#aimessage) for more information on what the response type may look like.

Once the tools are chosen, you will usually want to invoke them and then pass the results back to the model so that it can complete whatever task
it's performing.

There are generally two different ways to invoke the tool and pass back the response:

#### Invoke with just the arguments

When you invoke a tool with just the arguments, you will get back the raw tool output (usually a string).
Here's what this looks like:

```ts
import { ToolMessage } from "@langchain/core/messages";

const toolCall = aiMessage.tool_calls[0]; // ToolCall(args={...}, id=..., ...)
const toolOutput = await tool.invoke(toolCall.args);
const toolMessage = new ToolMessage({
  content: toolOutput,
  name: toolCall.name,
  tool_call_id: toolCall.id,
});
```

Note that the `content` field will generally be passed back to the model.
If you do not want the raw tool response to be passed to the model, but you still want to keep it around,
you can transform the tool output but also pass it as an artifact (read more about [`ToolMessage.artifact` here](/docs/concepts/#toolmessage))

```ts
// Same code as above
const responseForModel = someTransformation(response);
const toolMessage = new ToolMessage({
  content: responseForModel,
  tool_call_id: toolCall.id,
  name: toolCall.name,
  artifact: response,
});
```

#### Invoke with `ToolCall`

The other way to invoke a tool is to call it with the full `ToolCall` that was generated by the model.
When you do this, the tool will return a `ToolMessage`.
The benefits of this are that you don't have to write the logic yourself to transform the tool output into a ToolMessage.
Here's what this looks like:

```ts
const toolCall = aiMessage.tool_calls[0];
const toolMessage = await tool.invoke(toolCall);
```

If you are invoking the tool this way and want to include an [artifact](/docs/concepts/#toolmessage) for the `ToolMessage`, you will need to have the tool return a tuple
with two items: the `content` and the `artifact`.
Read more about [defining tools that return artifacts here](/docs/how_to/tool_artifacts/).

#### Best practices

When designing tools to be used by a model, it is important to keep in mind that:

- Chat models that have explicit [tool-calling APIs](/docs/concepts/#functiontool-calling) will be better at tool calling than non-fine-tuned models.
- Models will perform better if the tools have well-chosen names, descriptions, and JSON schemas. This another form of prompt engineering.
- Simple, narrowly scoped tools are easier for models to use than complex tools.

#### Related

For specifics on how to use tools, see the [tools how-to guides](/docs/how_to/#tools).

To use a pre-built tool, see the [tool integration docs](/docs/integrations/tools/).

### Toolkits

Toolkits are collections of tools that are designed to be used together for specific tasks. They have convenient loading methods.

All Toolkits expose a `getTools` method which returns an array of tools.
You can therefore do:

```typescript
// Initialize a toolkit
const toolkit = new ExampleTookit(...)

// Get list of tools
const tools = toolkit.getTools()
```

### Agents

By themselves, language models can't take actions - they just output text.
A big use case for LangChain is creating **agents**.
Agents are systems that use an LLM as a reasoning engineer to determine which actions to take and what the inputs to those actions should be.
The results of those actions can then be fed back into the agent and it determine whether more actions are needed, or whether it is okay to finish.

[LangGraph](https://github.com/langchain-ai/langgraphjs) is an extension of LangChain specifically aimed at creating highly controllable and customizable agents.
Please check out that [documentation](https://langchain-ai.github.io/langgraphjs/) for a more in depth overview of agent concepts.

There is a legacy agent concept in LangChain that we are moving towards deprecating: `AgentExecutor`.
AgentExecutor was essentially a runtime for agents.
It was a great place to get started, however, it was not flexible enough as you started to have more customized agents.
In order to solve that we built LangGraph to be this flexible, highly-controllable runtime.

If you are still using AgentExecutor, do not fear: we still have a guide on [how to use AgentExecutor](/docs/how_to/agent_executor).
It is recommended, however, that you start to transition to [LangGraph](https://github.com/langchain-ai/langgraphjs).
In order to assist in this we have put together a [transition guide on how to do so](/docs/how_to/migrate_agent).

#### ReAct agents

One popular architecture for building agents is [**ReAct**](https://arxiv.org/abs/2210.03629).
ReAct combines reasoning and acting in an iterative process - in fact the name "ReAct" stands for "Reason" and "Act".

The general flow looks like this:

- The model will "think" about what step to take in response to an input and any previous observations.
- The model will then choose an action from available tools (or choose to respond to the user).
- The model will generate arguments to that tool.
- The agent runtime (executor) will parse out the chosen tool and call it with the generated arguments.
- The executor will return the results of the tool call back to the model as an observation.
- This process repeats until the agent chooses to respond.

There are general prompting based implementations that do not require any model-specific features, but the most
reliable implementations use features like [tool calling](/docs/how_to/tool_calling/) to reliably format outputs
and reduce variance.

Please see the [LangGraph documentation](https://langchain-ai.github.io/langgraph/) for more information,
or [this how-to guide](/docs/how_to/migrate_agent/) for specific information on migrating to LangGraph.

### Callbacks

LangChain provides a callbacks system that allows you to hook into the various stages of your LLM application. This is useful for logging, monitoring, streaming, and other tasks.

You can subscribe to these events by using the `callbacks` argument available throughout the API. This argument is list of handler objects, which are expected to implement one or more of the methods described below in more detail.

#### Callback Events


| Event            | Event Trigger                               | Associated Method      |
| ---------------- | ------------------------------------------- | ---------------------- |
| Chat model start | When a chat model starts                    | `handleChatModelStart` |
| LLM start        | When a llm starts                           | `handleLLMStart`       |
| LLM new token    | When an llm OR chat model emits a new token | `handleLLMNewToken`    |
| LLM ends         | When an llm OR chat model ends              | `handleLLMEnd`         |
| LLM errors       | When an llm OR chat model errors            | `handleLLMError`       |
| Chain start      | When a chain starts running                 | `handleChainStart`     |
| Chain end        | When a chain ends                           | `handleChainEnd`       |
| Chain error      | When a chain errors                         | `handleChainError`     |
| Tool start       | When a tool starts running                  | `handleToolStart`      |
| Tool end         | When a tool ends                            | `handleToolEnd`        |
| Tool error       | When a tool errors                          | `handleToolError`      |
| Agent action     | When an agent takes an action               | `handleAgentAction`    |
| Agent finish     | When an agent ends                          | `handleAgentEnd`       |
| Retriever start  | When a retriever starts                     | `handleRetrieverStart` |
| Retriever end    | When a retriever ends                       | `handleRetrieverEnd`   |
| Retriever error  | When a retriever errors                     | `handleRetrieverError` |
| Text             | When arbitrary text is run                  | `handleText`           |

#### Callback handlers

`CallbackHandlers` are objects that implement the [`CallbackHandler`](https://api.js.langchain.com/interfaces/langchain_core.callbacks_base.CallbackHandlerMethods.html) interface, which has a method for each event that can be subscribed to.
The `CallbackManager` will call the appropriate method on each handler when the event is triggered.

#### Passing callbacks

The `callbacks` property is available on most objects throughout the API (Models, Tools, Agents, etc.) in two different places:

- **Request callbacks**: Passed at the time of the request in addition to the input data.
  Available on all standard `Runnable` objects. These callbacks are INHERITED by all children
  of the object they are defined on. For example, `chain.invoke({foo: "bar"}, {callbacks: [handler]})`.
- **Constructor callbacks**: defined in the constructor, e.g. `new ChatAnthropic({ callbacks: [handler], tags: ["a-tag"] })`. In this case, the callbacks will be used for all calls made on that object, and will be scoped to that object only.
  For example, if you initialize a chat model with constructor callbacks, then use it within a chain, the callbacks will only be invoked for calls to that model.

:::warning
Constructor callbacks are scoped only to the object they are defined on. They are **not** inherited by children
of the object.
:::

If you're creating a custom chain or runnable, you need to remember to propagate request time
callbacks to any child objects.

For specifics on how to use callbacks, see the [relevant how-to guides here](/docs/how_to/#callbacks).

## Techniques

### Streaming

Individual LLM calls often run for much longer than traditional resource requests.
This compounds when you build more complex chains or agents that require multiple reasoning steps.

Fortunately, LLMs generate output iteratively, which means it's possible to show sensible intermediate results
before the final response is ready. Consuming output as soon as it becomes available has therefore become a vital part of the UX
around building apps with LLMs to help alleviate latency issues, and LangChain aims to have first-class support for streaming.

Below, we'll discuss some concepts and considerations around streaming in LangChain.

#### `.stream()`

Most modules in LangChain include the `.stream()` method as an ergonomic streaming interface.
`.stream()` returns an iterator, which you can consume with a [`for await...of`](https://developer.mozilla.org/en-US/docs/Web/JavaScript/Reference/Statements/for-await...of) loop. Here's an example with a chat model:

```ts
import { ChatAnthropic } from "@langchain/anthropic";
import { concat } from "@langchain/core/utils/stream";
import type { AIMessageChunk } from "@langchain/core/messages";

const model = new ChatAnthropic({ model: "claude-3-sonnet-20240229" });

const stream = await model.stream("what color is the sky?");

let gathered: AIMessageChunk | undefined = undefined;

for await (const chunk of stream) {
  console.log(chunk);
  if (gathered === undefined) {
    gathered = chunk;
  } else {
    gathered = concat(gathered, chunk);
  }
}

console.log(gathered);
```

For models (or other components) that don't support streaming natively, this iterator would just yield a single chunk, but
you could still use the same general pattern when calling them. Using `.stream()` will also automatically call the model in streaming mode
without the need to provide additional config.

The type of each outputted chunk depends on the type of component - for example, chat models yield [`AIMessageChunks`](https://api.js.langchain.com/classes/langchain_core.messages.AIMessageChunk.html).
Because this method is part of [LangChain Expression Language](/docs/concepts/#langchain-expression-language),
you can handle formatting differences from different outputs using an [output parser](/docs/concepts/#output-parsers) to transform
each yielded chunk.

You can check out [this guide](/docs/how_to/streaming/#using-stream) for more detail on how to use `.stream()`.

#### `.streamEvents()`

While the `.stream()` method is intuitive, it can only return the final generated value of your chain. This is fine for single LLM calls,
but as you build more complex chains of several LLM calls together, you may want to use the intermediate values of
the chain alongside the final output - for example, returning sources alongside the final generation when building a chat
over documents app.

There are ways to do this [using callbacks](/docs/concepts/#callbacks-1), or by constructing your chain in such a way that it passes intermediate
values to the end with something like chained [`.assign()`](/docs/how_to/passthrough/) calls, but LangChain also includes an
`.streamEvents()` method that combines the flexibility of callbacks with the ergonomics of `.stream()`. When called, it returns an iterator
which yields [various types of events](/docs/how_to/streaming/#event-reference) that you can filter and process according
to the needs of your project.

Here's one small example that prints just events containing streamed chat model output:

```ts
import { StringOutputParser } from "@langchain/core/output_parsers";
import { ChatPromptTemplate } from "@langchain/core/prompts";
import { ChatAnthropic } from "@langchain/anthropic";

const model = new ChatAnthropic({ model: "claude-3-sonnet-20240229" });

const prompt = ChatPromptTemplate.fromTemplate("tell me a joke about {topic}");
const parser = new StringOutputParser();
const chain = prompt.pipe(model).pipe(parser);

const eventStream = await chain.streamEvents(
  { topic: "parrot" },
  { version: "v2" }
);

for await (const event of eventStream) {
  const kind = event.event;
  if (kind === "on_chat_model_stream") {
    console.log(event);
  }
}
```

You can roughly think of it as an iterator over callback events (though the format differs) - and you can use it on almost all LangChain components!

See [this guide](/docs/how_to/streaming/#using-stream-events) for more detailed information on how to use `.streamEvents()`,
or [this guide](/docs/how_to/callbacks_custom_events) for how to stream custom events from within a chain.

#### Callbacks

The lowest level way to stream outputs from LLMs in LangChain is via the [callbacks](/docs/concepts/#callbacks) system. You can pass a
callback handler that handles the [`handleLLMNewToken`](https://api.js.langchain.com/interfaces/langchain_core.callbacks_base.CallbackHandlerMethods.html#handleLLMNewToken) event into LangChain components. When that component is invoked, any
[LLM](/docs/concepts/#llms) or [chat model](/docs/concepts/#chat-models) contained in the component calls
the callback with the generated token. Within the callback, you could pipe the tokens into some other destination, e.g. a HTTP response.
You can also handle the [`handleLLMEnd`](https://api.js.langchain.com/interfaces/langchain_core.callbacks_base.CallbackHandlerMethods.html#handleLLMEnd) event to perform any necessary cleanup.

You can see [this how-to section](/docs/how_to/#callbacks) for more specifics on using callbacks.

Callbacks were the first technique for streaming introduced in LangChain. While powerful and generalizable,
they can be unwieldy for developers. For example:


- You need to explicitly initialize and manage some aggregator or other stream to collect results.
- The execution order isn't explicitly guaranteed, and you could theoretically have a callback run after the `.invoke()` method finishes.
- Providers would often make you pass an additional parameter to stream outputs instead of returning them all at once.
- You would often ignore the result of the actual model call in favor of callback results.

#### Tokens

The unit that most model providers use to measure input and output is via a unit called a **token**.
Tokens are the basic units that language models read and generate when processing or producing text.
The exact definition of a token can vary depending on the specific way the model was trained -
for instance, in English, a token could be a single word like "apple", or a part of a word like "app".

When you send a model a prompt, the words and characters in the prompt are encoded into tokens using a **tokenizer**.
The model then streams back generated output tokens, which the tokenizer decodes into human-readable text.
The below example shows how OpenAI models tokenize `LangChain is cool!`:

![](/img/tokenization.png)

You can see that it gets split into 5 different tokens, and that the boundaries between tokens are not exactly the same as word boundaries.

The reason language models use tokens rather than something more immediately intuitive like "characters"
has to do with how they process and understand text. At a high-level, language models iteratively predict their next generated output based on
the initial input and their previous generations. Training the model using tokens language models to handle linguistic
units (like words or subwords) that carry meaning, rather than individual characters, which makes it easier for the model
to learn and understand the structure of the language, including grammar and context.
Furthermore, using tokens can also improve efficiency, since the model processes fewer units of text compared to character-level processing.

### Function/tool calling

:::info
We use the term tool calling interchangeably with function calling. Although
function calling is sometimes meant to refer to invocations of a single function,
we treat all models as though they can return multiple tool or function calls in
each message.
:::

Tool calling allows a [chat model](/docs/concepts/#chat-models) to respond to a given prompt by generating output that
matches a user-defined schema.

While the name implies that the model is performing
some action, this is actually not the case! The model only generates the arguments to a tool, and actually running the tool (or not) is up to the user.
One common example where you **wouldn't** want to call a function with the generated arguments
is if you want to [extract structured output matching some schema](/docs/concepts/#structured-output)
from unstructured text. You would give the model an "extraction" tool that takes
parameters matching the desired schema, then treat the generated output as your final
result.

![Diagram of a tool call by a chat model](/img/tool_call.png)

Tool calling is not universal, but is supported by many popular LLM providers, including [Anthropic](/docs/integrations/chat/anthropic/),
[Cohere](/docs/integrations/chat/cohere/), [Google](/docs/integrations/chat/google_vertex_ai/),
[Mistral](/docs/integrations/chat/mistral/), [OpenAI](/docs/integrations/chat/openai/), and even for locally-running models via [Ollama](/docs/integrations/chat/ollama/).

LangChain provides a standardized interface for tool calling that is consistent across different models.

The standard interface consists of:

- `ChatModel.bind_tools()`: a method for specifying which tools are available for a model to call. This method accepts [LangChain tools](/docs/concepts/#tools) as well as model-specific formats.
- `AIMessage.tool_calls`: an attribute on the `AIMessage` returned from the model for accessing the tool calls requested by the model.

#### Tool usage

After the model calls tools, you can use the tool by invoking it, then passing the arguments back to the model.
LangChain provides the [`Tool`](/docs/concepts/#tools) abstraction to help you handle this.

The general flow is this:

1. Generate tool calls with a chat model in response to a query.
2. Invoke the appropriate tools using the generated tool call as arguments.
3. Format the result of the tool invocations as [`ToolMessages`](/docs/concepts/#toolmessage).
4. Pass the entire list of messages back to the model so that it can generate a final answer (or call more tools).

![Diagram of a complete tool calling flow](/img/tool_calling_flow.png)

This is how tool calling [agents](/docs/concepts/#agents) perform tasks and answer queries.

Check out some more focused guides below:

- [How to use chat models to call tools](/docs/how_to/tool_calling/)
- [How to pass tool outputs to chat models](/docs/how_to/tool_results_pass_to_model/)
- [Building an agent with LangGraph](https://langchain-ai.github.io/langgraphjs/tutorials/introduction/)

### Structured output

LLMs are capable of generating arbitrary text. This enables the model to respond appropriately to a wide
range of inputs, but for some use-cases, it can be useful to constrain the LLM's output
to a specific format or structure. This is referred to as **structured output**.

For example, if the output is to be stored in a relational database,
it is much easier if the model generates output that adheres to a defined schema or format.
[Extracting specific information](/docs/tutorials/extraction/) from unstructured text is another
case where this is particularly useful. Most commonly, the output format will be JSON,
though other formats such as [XML](/docs/how_to/output_parser_xml/) can be useful too. Below, we'll discuss
a few ways to get structured output from models in LangChain.

#### `.withStructuredOutput()`

For convenience, some LangChain chat models support a [`.withStructuredOutput()`](/docs/how_to/structured_output/#the-.withstructuredoutput-method) method.
This method only requires a schema as input, and returns an object matching the requested schema.
Generally, this method is only present on models that support one of the more advanced methods described below,
and will use one of them under the hood. It takes care of importing a suitable output parser and
formatting the schema in the right format for the model.

Here's an example:

```ts
import { z } from "zod";

const joke = z.object({
  setup: z.string().describe("The setup of the joke"),
  punchline: z.string().describe("The punchline to the joke"),
  rating: z.number().optional().describe("How funny the joke is, from 1 to 10"),
});

// Can also pass in JSON schema.
// It's also beneficial to pass in an additional "name" parameter to give the
// model more context around the type of output to generate.
const structuredLlm = model.withStructuredOutput(joke);

await structuredLlm.invoke("Tell me a joke about cats");
```

```
{
  setup: "Why don't cats play poker in the wild?",
  punchline: "Too many cheetahs.",
  rating: 7
}
```

We recommend this method as a starting point when working with structured output:

- It uses other model-specific features under the hood, without the need to import an output parser.
- For the models that use tool calling, no special prompting is needed.
- If multiple underlying techniques are supported, you can supply a `method` parameter to
  [toggle which one is used](/docs/how_to/structured_output/#specifying-the-output-method-advanced).

You may want or need to use other techiniques if:

- The chat model you are using does not support tool calling.
- You are working with very complex schemas and the model is having trouble generating outputs that conform.

For more information, check out this [how-to guide](/docs/how_to/structured_output/#the-.withstructuredoutput-method).

You can also check out [this table](/docs/integrations/chat/) for a list of models that support
`.withStructuredOutput()`.

#### Raw prompting

The most intuitive way to get a model to structure output is to ask nicely.
In addition to your query, you can give instructions describing what kind of output you'd like, then
parse the output using an [output parser](/docs/concepts/#output-parsers) to convert the raw
model message or string output into something more easily manipulated.

The biggest benefit to raw prompting is its flexibility:

- Raw prompting does not require any special model features, only sufficient reasoning capability to understand
  the passed schema.
- You can prompt for any format you'd like, not just JSON. This can be useful if the model you
  are using is more heavily trained on a certain type of data, such as XML or YAML.

However, there are some drawbacks too:

- LLMs are non-deterministic, and prompting a LLM to consistently output data in the exactly correct format
  for smooth parsing can be surprisingly difficult and model-specific.
- Individual models have quirks depending on the data they were trained on, and optimizing prompts can be quite difficult.
  Some may be better at interpreting [JSON schema](https://json-schema.org/), others may be best with TypeScript definitions,
  and still others may prefer XML.

While features offered by model providers may increase reliability, prompting techniques remain important for tuning your
results no matter which method you choose.

#### JSON mode


Some models, such as [Mistral](/docs/integrations/chat/mistral/), [OpenAI](/docs/integrations/chat/openai/),
[Together AI](/docs/integrations/chat/togetherai/) and [Ollama](/docs/integrations/chat/ollama/),
support a feature called **JSON mode**, usually enabled via config.

When enabled, JSON mode will constrain the model's output to always be some sort of valid JSON.
Often they require some custom prompting, but it's usually much less burdensome than completely raw prompting and
more along the lines of,
`"you must always return JSON"`. The [output also is generally easier to parse](/docs/how_to/output_parser_json/).

It's also generally simpler to use directly and more commonly available than tool calling, and can give
more flexibility around prompting and shaping results than tool calling.

Here's an example:

```ts
import { JsonOutputParser } from "@langchain/core/output_parsers";
import { ChatPromptTemplate } from "@langchain/core/prompts";
import { ChatOpenAI } from "@langchain/openai";

const model = new ChatOpenAI({
  model: "gpt-4o",
  modelKwargs: {
    response_format: { type: "json_object" },
  },
});

const TEMPLATE = `Answer the user's question to the best of your ability.
You must always output a JSON object with an "answer" key and a "followup_question" key.

{question}`;

const prompt = ChatPromptTemplate.fromTemplate(TEMPLATE);

const chain = prompt.pipe(model).pipe(new JsonOutputParser());

await chain.invoke({ question: "What is the powerhouse of the cell?" });
```

```
{
  answer: "The powerhouse of the cell is the mitochondrion.",
  followup_question: "Would you like to learn more about the functions of mitochondria?"
}
```

For a full list of model providers that support JSON mode, see [this table](/docs/integrations/chat/).

#### Tool calling {#structured-output-tool-calling}

For models that support it, [tool calling](/docs/concepts/#functiontool-calling) can be very convenient for structured output. It removes the
guesswork around how best to prompt schemas in favor of a built-in model feature.

It works by first binding the desired schema either directly or via a [LangChain tool](/docs/concepts/#tools) to a
[chat model](/docs/concepts/#chat-models) using the `.bind_tools()` method. The model will then generate an `AIMessage` containing
a `tool_calls` field containing `args` that match the desired shape.

There are several acceptable formats you can use to bind tools to a model in LangChain. Here's one example using [Zod](https://zod.dev):

```ts
import { z } from "zod";
import { zodToJsonSchema } from "zod-to-json-schema";
import { ChatOpenAI } from "@langchain/openai";

const toolSchema = z.object({
  answer: z.string().describe("The answer to the user's question"),
  followup_question: z
    .string()
    .describe("A followup question the user could ask"),
});

const model = new ChatOpenAI({
  model: "gpt-4o",
  temperature: 0,
});

const modelWithTools = model.bindTools([
  {
    type: "function",
    function: {
      name: "response_formatter",
      description:
        "Always use this tool to structure your response to the user.",
      parameters: zodToJsonSchema(toolSchema),
    },
  },
]);

const aiMessage = await modelWithTools.invoke(
  "What is the powerhouse of the cell?"
);

aiMessage.tool_calls?.[0].args;
```

```
{
  answer: 'The powerhouse of the cell is the mitochondrion.',
  followup_question: 'What is the main function of the mitochondrion in the cell?'
}
```

Tool calling is a generally consistent way to get a model to generate structured output, and is the default technique
used for the [`.withStructuredOutput()`](/docs/concepts/#withstructuredoutput) method when a model supports it.

The following how-to guides are good practical resources for using function/tool calling for structured output:

- [How to return structured data from an LLM](/docs/how_to/structured_output/)
- [How to use a model to call tools](/docs/how_to/tool_calling)

### Few-shot prompting

One of the most effective ways to improve model performance is to give a model examples of what you want it to do. The technique of adding example inputs and expected outputs to a model prompt is known as "few-shot prompting". There are a few things to think about when doing few-shot prompting:

1. How are examples generated?
2. How many examples are in each prompt?
3. How are examples selected at runtime?
4. How are examples formatted in the prompt?

Here are the considerations for each.

#### 1. Generating examples

The first and most important step of few-shot prompting is coming up with a good dataset of examples.
Good examples should be relevant at runtime, clear, informative, and provide information that was not already known to the model.

At a high-level, the basic ways to generate examples are:

- Manual: a person/people generates examples they think are useful.
- Better model: a better (presumably more expensive/slower) model's responses are used as examples for a worse (presumably cheaper/faster) model.
- User feedback: users (or labelers) leave feedback on interactions with the application and examples are generated based on that feedback (for example, all interactions with positive feedback could be turned into examples).
- LLM feedback: same as user feedback but the process is automated by having models evaluate themselves.

Which approach is best depends on your task. For tasks where a small number core principles need to be understood really well, it can be valuable hand-craft a few really good examples.
For tasks where the space of correct behaviors is broader and more nuanced, it can be useful to generate many examples in a more automated fashion so that there's a higher likelihood of there being some highly relevant examples for any runtime input.

**Single-turn v.s. multi-turn examples**

Another dimension to think about when generating examples is what the example is actually showing.

The simplest types of examples just have a user input and an expected model output. These are single-turn examples.

One more complex type if example is where the example is an entire conversation, usually in which a model initially responds incorrectly and a user then tells the model how to correct its answer.
This is called a multi-turn example. Multi-turn examples can be useful for more nuanced tasks where its useful to show common errors and spell out exactly why they're wrong and what should be done instead.

#### 2. Number of examples

Once we have a dataset of examples, we need to think about how many examples should be in each prompt.
The key tradeoff is that more examples generally improve performance, but larger prompts increase costs and latency.
And beyond some threshold having too many examples can start to confuse the model.
Finding the right number of examples is highly dependent on the model, the task, the quality of the examples, and your cost and latency constraints.
Anecdotally, the better the model is the fewer examples it needs to perform well and the more quickly you hit steeply diminishing returns on adding more examples.
But, the best/only way to reliably answer this question is to run some experiments with different numbers of examples.

#### 3. Selecting examples

Assuming we are not adding our entire example dataset into each prompt, we need to have a way of selecting examples from our dataset based on a given input. We can do this:

- Randomly
- By (semantic or keyword-based) similarity of the inputs
- Based on some other constraints, like token size

LangChain has a number of [`ExampleSelectors`](/docs/concepts/#example-selectors) which make it easy to use any of these techniques.

Generally, selecting by semantic similarity leads to the best model performance. But how important this is is again model and task specific, and is something worth experimenting with.

#### 4. Formatting examples

Most state-of-the-art models these days are chat models, so we'll focus on formatting examples for those. Our basic options are to insert the examples:

- In the system prompt as a string
- As their own messages

If we insert our examples into the system prompt as a string, we'll need to make sure it's clear to the model where each example begins and which parts are the input versus output. Different models respond better to different syntaxes, like [ChatML](https://learn.microsoft.com/en-us/azure/ai-services/openai/how-to/chat-markup-language), XML, TypeScript, etc.

If we insert our examples as messages, where each example is represented as a sequence of Human, AI messages, we might want to also assign [names](/docs/concepts/#messages) to our messages like `"exampleUser"` and `"exampleAssistant"` to make it clear that these messages correspond to different actors than the latest input message.

**Formatting tool call examples**

One area where formatting examples as messages can be tricky is when our example outputs have tool calls. This is because different models have different constraints on what types of message sequences are allowed when any tool calls are generated.

- Some models require that any AIMessage with tool calls be immediately followed by ToolMessages for every tool call,
- Some models additionally require that any ToolMessages be immediately followed by an AIMessage before the next HumanMessage,
- Some models require that tools are passed in to the model if there are any tool calls / ToolMessages in the chat history.


These requirements are model-specific and should be checked for the model you are using. If your model requires ToolMessages after tool calls and/or AIMessages after ToolMessages and your examples only include expected tool calls and not the actual tool outputs, you can try adding dummy ToolMessages / AIMessages to the end of each example with generic contents to satisfy the API constraints.

In these cases it's especially worth experimenting with inserting your examples as strings versus messages, as having dummy messages can adversely affect certain models.

You can see a case study of how Anthropic and OpenAI respond to different few-shot prompting techniques on two different tool calling benchmarks [here](https://blog.langchain.dev/few-shot-prompting-to-improve-tool-calling-performance/).

### Retrieval

LLMs are trained on a large but fixed dataset, limiting their ability to reason over private or recent information. Fine-tuning an LLM with specific facts is one way to mitigate this, but is often [poorly suited for factual recall](https://www.anyscale.com/blog/fine-tuning-is-for-form-not-facts) and [can be costly](https://www.glean.com/blog/how-to-build-an-ai-assistant-for-the-enterprise).
Retrieval is the process of providing relevant information to an LLM to improve its response for a given input. Retrieval augmented generation (RAG) is the process of grounding the LLM generation (output) using the retrieved information.

:::tip

- See our RAG from Scratch [video series](https://youtube.com/playlist?list=PLfaIDFEXuae2LXbO1_PKyVJiQ23ZztA0x&feature=shared).
  The code examples are in Python but is useful for a general overview of RAG concepts for visual learners.
- For a high-level guide on retrieval, see this [tutorial on RAG](/docs/tutorials/rag/).

:::

RAG is only as good as the retrieved documents’ relevance and quality. Fortunately, an emerging set of techniques can be employed to design and improve RAG systems. We've focused on taxonomizing and summarizing many of these techniques (see below figure) and will share some high-level strategic guidance in the following sections.
You can and should experiment with using different pieces together. You might also find [this LangSmith guide](https://docs.smith.langchain.com/how_to_guides/evaluation/evaluate_llm_application) useful for showing how to evaluate different iterations of your app.

![](/img/rag_landscape.png)

#### Query Translation

First, consider the user input(s) to your RAG system. Ideally, a RAG system can handle a wide range of inputs, from poorly worded questions to complex multi-part queries.
**Using an LLM to review and optionally modify the input is the central idea behind query translation.** This serves as a general buffer, optimizing raw user inputs for your retrieval system.
For example, this can be as simple as extracting keywords or as complex as generating multiple sub-questions for a complex query.

| Name                                                                                | When to use                                                                     | Description                                                                                |
| --------------------------------------------------------------------------------------------------------------------------- | ------------------------------------------------------------------------------- | ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
| [Multi-query](/docs/how_to/multiple_queries/)                                                                               | When you need to cover multiple perspectives of a question.                     | Rewrite the user question from multiple perspectives, retrieve documents for each rewritten question, return the unique documents for all queries.                                                                                |
| [Decomposition (Python cookbook)](https://github.com/langchain-ai/rag-from-scratch/blob/main/rag_from_scratch_5_to_9.ipynb) | When a question can be broken down into smaller subproblems.                    | Decompose a question into a set of subproblems / questions, which can either be solved sequentially (use the answer from first + retrieval to answer the second) or in parallel (consolidate each answer into final answer).                |
| [Step-back (Python cookbook)](https://github.com/langchain-ai/rag-from-scratch/blob/main/rag_from_scratch_5_to_9.ipynb)     | When a higher-level conceptual understanding is required.                       | First prompt the LLM to ask a generic step-back question about higher-level concepts or principles, and retrieve relevant facts about them. Use this grounding to help answer the user question.                                            |
| [HyDE (Python cookbook)](https://github.com/langchain-ai/rag-from-scratch/blob/main/rag_from_scratch_5_to_9.ipynb)          | If you have challenges retrieving relevant documents using the raw user inputs. | Use an LLM to convert questions into hypothetical documents that answer the question. Use the embedded hypothetical documents to retrieve real documents with the premise that doc-doc similarity search can produce more relevant matches. |

:::tip

See our Python RAG from Scratch videos for a few different specific approaches:

- [Multi-query](https://youtu.be/JChPi0CRnDY?feature=shared)
- [Decomposition](https://youtu.be/h0OPWlEOank?feature=shared)
- [Step-back](https://youtu.be/xn1jEjRyJ2U?feature=shared)
- [HyDE](https://youtu.be/SaDzIVkYqyY?feature=shared)

:::

#### Routing

Second, consider the data sources available to your RAG system. You want to query across more than one database or across structured and unstructured data sources. **Using an LLM to review the input and route it to the appropriate data source is a simple and effective approach for querying across sources.**

| Name                                                                     | When to use                                                                         | Description                                                                                |
| ------------------------------------------------------------------------ | ----------------------------------------------------------------------------------- | --------------------------------------------------------------------------------------------------------------------------------- |
| [Logical routing](/docs/how_to/routing/)                                 | When you can prompt an LLM with rules to decide where to route the input.           | Logical routing can use an LLM to reason about the query and choose which datastore is most appropriate.                          |
| [Semantic routing](/docs/how_to/routing/#routing-by-semantic-similarity) | When semantic similarity is an effective way to determine where to route the input. | Semantic routing embeds both query and, typically a set of prompts. It then chooses the appropriate prompt based upon similarity. |

:::tip

See our Python RAG from Scratch video on [routing](https://youtu.be/pfpIndq7Fi8?feature=shared).

:::

#### Query Construction

Third, consider whether any of your data sources require specific query formats. Many structured databases use SQL. Vector stores often have specific syntax for applying keyword filters to document metadata. **Using an LLM to convert a natural language query into a query syntax is a popular and powerful approach.**
In particular, [text-to-SQL](/docs/tutorials/sql_qa/), [text-to-Cypher](/docs/tutorials/graph/), and [query analysis for metadata filters](/docs/tutorials/query_analysis/#query-analysis) are useful ways to interact with structured, graph, and vector databases respectively.

| Name                                     | When to Use                                                                                | Description                                                                                |
| ---------------------------------------- | ------------------------------------------------------------------------------------------------------------------------------------ | ---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
| [Text to SQL](/docs/tutorials/sql_qa/)   | If users are asking questions that require information housed in a relational database, accessible via SQL.                          | This uses an LLM to transform user input into a SQL query.                                                                                |
| [Text-to-Cypher](/docs/tutorials/graph/) | If users are asking questions that require information housed in a graph database, accessible via Cypher.                            | This uses an LLM to transform user input into a Cypher query.                                                                                |
| [Self Query](/docs/how_to/self_query/)   | If users are asking questions that are better answered by fetching documents based on metadata rather than similarity with the text. | This uses an LLM to transform user input into two things: (1) a string to look up semantically, (2) a metadata filter to go along with it. This is useful because oftentimes questions are about the METADATA of documents (not the content itself). |

:::tip

See our [blog post overview](https://blog.langchain.dev/query-construction/) and RAG from Scratch video on [query construction](https://youtu.be/kl6NwWYxvbM?feature=shared), the process of text-to-DSL where DSL is a domain specific language required to interact with a given database. This converts user questions into structured queries.

:::

#### Indexing

Fouth, consider the design of your document index. A simple and powerful idea is to **decouple the documents that you index for retrieval from the documents that you pass to the LLM for generation.** Indexing frequently uses embedding models with vector stores, which [compress the semantic information in documents to fixed-size vectors](/docs/concepts/#embedding-models).

Many RAG approaches focus on splitting documents into chunks and retrieving some number based on similarity to an input question for the LLM. But chunk size and chunk number can be difficult to set and affect results if they do not provide full context for the LLM to answer a question. Furthermore, LLMs are increasingly capable of processing millions of tokens.


:::tip
We're unifying model params across all packages. We now suggest using `model` instead of `modelName`, and `apiKey` for API keys.
:::


---
sidebar_position: 1
sidebar_label: v0.2
---

# LangChain v0.2

LangChain v0.2 was released in May 2024. This release includes a number of breaking changes and deprecations. This document contains a guide on upgrading to 0.2.x, as well as a list of deprecations and breaking changes.

:::note Reference

- [Migrating to Astream Events v2](/docs/versions/v0_2/migrating_astream_events)

:::

## Migration

This documentation will help you upgrade your code to LangChain `0.2.x.`. To prepare for migration, we first recommend you take the following steps:

1. install the 0.2.x versions of `@langchain/core`, langchain and upgrade to recent versions of other packages that you may be using (e.g. `@langchain/langgraph`, `@langchain/community`, `@langchain/openai`, etc.)
2. Verify that your code runs properly with the new packages (e.g., unit tests pass)
3. Install a recent version of `langchain-cli` , and use the tool to replace old imports used by your code with the new imports. (See instructions below.)
4. Manually resolve any remaining deprecation warnings
5. Re-run unit tests

### Upgrade to new imports

We created a tool to help migrate your code. This tool is still in **beta** and may not cover all cases, but
we hope that it will help you migrate your code more quickly.

The migration script has the following limitations:

1. It's limited to helping users move from old imports to new imports. It doesn't help address other deprecations.
2. It can't handle imports that involve `as` .
3. New imports are always placed in global scope, even if the old import that was replaced was located inside some local scope (e..g, function body).
4. It will likely miss some deprecated imports.

Here is an example of the import changes that the migration script can help apply automatically:

| From Package           | To Package                 | Deprecated Import                                                          | New Import                                                                       |
| ---------------------- | -------------------------- | -------------------------------------------------------------------------- | -------------------------------------------------------------------------------- |
| `langchain`            | `@langchain/community`     | `import { UpstashVectorStore } from "langchain/vectorstores/upstash"`      | `import { UpstashVectorStore } from "@langchain/community/vectorstores/upstash"` |
| `@langchain/community` | `@langchain/openai`        | `import { ChatOpenAI } from "@langchain/community/chat_models/openai"`     | `import { ChatOpenAI } from "@langchain/openai"`                                 |
| `langchain`            | `@langchain/core`          | `import { Document } from "langchain/schema/document"`                     | `import { Document } from "@langchain/core/documents"`                           |
| `langchain`            | `@langchain/textsplitters` | `import { RecursiveCharacterTextSplitter } from "langchain/text_splitter"` | `import { RecursiveCharacterTextSplitter } from "@langchain/textsplitters"`      |

#### Deprecation timeline

We have two main types of deprecations:

1. Code that was moved from `langchain` into another package (e.g, `@langchain/community`)

If you try to import it from `langchain`, it will fail since the entrypoint has been removed.

2. Code that has better alternatives available and will eventually be removed, so there's only a single way to do things. (e.g., `predictMessages` method in ChatModels has been deprecated in favor of `invoke`).

Many of these were marked for removal in 0.2. We have bumped the removal to 0.3.

#### Installation

:::note
The 0.2.X migration script is only available in version `0.0.14-rc.1` or later.
:::

```bash npm2yarn
npm i @langchain/scripts@0.0.14-rc.1
```

#### Usage

Given that the migration script is not perfect, you should make sure you have a backup of your code first (e.g., using version control like `git`).

For example, say your code still uses `import ChatOpenAI from "@langchain/community/chat_models/openai";`:

Invoking the migration script will replace this import with `import ChatOpenAI from "@langchain/openai";`.

```typescript
import { updateEntrypointsFrom0_x_xTo0_2_x } from "@langchain/scripts/migrations";

const pathToMyProject = "..."; // This path is used in the following glob pattern: `${projectPath}/**/*.{ts,tsx,js,jsx}`.

updateEntrypointsFrom0_x_xTo0_2_x({
  projectPath: pathToMyProject,
  shouldLog: true,
});
```

#### Other options

```typescript
updateEntrypointsFrom0_x_xTo0_2_x({
  projectPath: pathToMyProject,
  tsConfigPath: "tsconfig.json", // Path to the tsConfig file. This will be used to load all the project files into the script.
  testRun: true, // If true, the script will not save any changes, but will log the changes that would be made.
  files: ["..."], // A list of .ts file paths to check. If this is provided, the script will only check these files.
});
```


{
 "cells": [
  {
   "cell_type": "markdown",
   "id": "ce8457ed-c0b1-4a74-abbd-9d3d2211270f",
   "metadata": {},
   "source": [
    "# Migrating off ConversationTokenBufferMemory\n",
    "\n",
    "Follow this guide if you're trying to migrate off one of the old memory classes listed below:\n",
    "\n",
    "\n",
    "| Memory Type                      | Description                                                                                |\n",
    "|----------------------------------|-------------------------------------------------------------------------------------------------------------------------------------------------------------------|\n",
    "| `ConversationTokenBufferMemory`  | Keeps only the most recent messages in the conversation under the constraint that the total number of tokens in the conversation does not exceed a certain limit. |\n",
    "\n",
    "`ConversationTokenBufferMemory` applies additional processing on top of the raw conversation history to trim the conversation history to a size that fits inside the context window of a chat model. \n",
    "\n",
    "This processing functionality can be accomplished using LangChain's built-in [trimMessages](https://api.js.langchain.com/functions/_langchain_core.messages.trimMessages.html) function."
   ]
  },
  {
   "cell_type": "markdown",
   "id": "79935247-acc7-4a05-a387-5d72c9c8c8cb",
   "metadata": {},
   "source": [
    "```{=mdx}\n",
    ":::important\n",
    "\n",
    "We’ll begin by exploring a straightforward method that involves applying processing logic to the entire conversation history.\n",
    "\n",
    "While this approach is easy to implement, it has a downside: as the conversation grows, so does the latency, since the logic is re-applied to all previous exchanges in the conversation at each turn.\n",
    "\n",
    "More advanced strategies focus on incrementally updating the conversation history to avoid redundant processing.\n",
    "\n",
    "For instance, the LangGraph [how-to guide on summarization](https://langchain-ai.github.io/langgraphjs/how-tos/add-summary-conversation-history/) demonstrates\n",
    "how to maintain a running summary of the conversation while discarding older messages, ensuring they aren't re-processed during later turns.\n",
    ":::\n",
    "```"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "d07f9459-9fb6-4942-99c9-64558aedd7d4",
   "metadata": {},
   "source": [
    "## Set up\n",
    "\n",
    "### Dependencies\n",
    "\n",
    "```{=mdx}\n",
    "import Npm2Yarn from \"@theme/Npm2Yarn\"\n",
    "\n",
    "<Npm2Yarn>\n",
    "  @langchain/openai @langchain/core zod\n",
    "</Npm2Yarn>\n",
    "```\n",
    "\n",
    "### Environment variables\n",
    "\n",
    "```typescript\n",
    "process.env.OPENAI_API_KEY = \"YOUR_OPENAI_API_KEY\";\n",
    "```\n",
    "\n",
    "```{=mdx}\n",
    "<details open>\n",
    "```"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "7ce2d951",
   "metadata": {},
   "source": [
    "## Reimplementing ConversationTokenBufferMemory logic\n",
    "\n",
    "Here, we'll use `trimMessages` to keeps the system message and the most recent messages in the conversation under the constraint that the total number of tokens in the conversation does not exceed a certain limit."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 1,
   "id": "e1550bee",
   "metadata": {},
   "outputs": [],
   "source": [
    "import {\n",
    "  AIMessage,\n",
    "  HumanMessage,\n",
    "  SystemMessage,\n",
    "} from \"@langchain/core/messages\";\n",
    "\n",
    "const messages = [\n",
    "  new SystemMessage(\"you're a good assistant, you always respond with a joke.\"),\n",
    "  new HumanMessage(\"i wonder why it's called langchain\"),\n",
    "  new AIMessage(\n",
    "    'Well, I guess they thought \"WordRope\" and \"SentenceString\" just didn\\'t have the same ring to it!'\n",
    "  ),\n",
    "  new HumanMessage(\"and who is harrison chasing anyways\"),\n",
    "  new AIMessage(\n",
    "      \"Hmmm let me think.\\n\\nWhy, he's probably chasing after the last cup of coffee in the office!\"\n",
    "  ),\n",
    "  new HumanMessage(\"why is 42 always the answer?\"),\n",
    "  new AIMessage(\n",
    "      \"Because it's the only number that's constantly right, even when it doesn't add up!\"\n",
    "  ),\n",
    "  new HumanMessage(\"What did the cow say?\"),\n",
    "]"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 2,
   "id": "6442f74b-2c36-48fd-a3d1-c7c5d18c050f",
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "SystemMessage {\n",
      "  \"content\": \"you're a good assistant, you always respond with a joke.\",\n",
      "  \"additional_kwargs\": {},\n",
      "  \"response_metadata\": {}\n",
      "}\n",
      "HumanMessage {\n",
      "  \"content\": \"and who is harrison chasing anyways\",\n",
      "  \"additional_kwargs\": {},\n",
      "  \"response_metadata\": {}\n",
      "}\n",
      "AIMessage {\n",
      "  \"content\": \"Hmmm let me think.\\n\\nWhy, he's probably chasing after the last cup of coffee in the office!\",\n",
      "  \"additional_kwargs\": {},\n",
      "  \"response_metadata\": {},\n",
      "  \"tool_calls\": [],\n",
      "  \"invalid_tool_calls\": []\n",
      "}\n",
      "HumanMessage {\n",
      "  \"content\": \"why is 42 always the answer?\",\n",
      "  \"additional_kwargs\": {},\n",
      "  \"response_metadata\": {}\n",
      "}\n",
      "AIMessage {\n",
      "  \"content\": \"Because it's the only number that's constantly right, even when it doesn't add up!\",\n",
      "  \"additional_kwargs\": {},\n",
      "  \"response_metadata\": {},\n",
      "  \"tool_calls\": [],\n",
      "  \"invalid_tool_calls\": []\n",
      "}\n",
      "HumanMessage {\n",
      "  \"content\": \"What did the cow say?\",\n",
      "  \"additional_kwargs\": {},\n",
      "  \"response_metadata\": {}\n",
      "}\n"
     ]
    }
   ],
   "source": [
    "import { trimMessages } from \"@langchain/core/messages\";\n",
    "import { ChatOpenAI } from \"@langchain/openai\";\n",
    "\n",
    "const selectedMessages = await trimMessages(\n",
    "  messages,\n",
    "  {\n",
    "    // Please see API reference for trimMessages for other ways to specify a token counter.\n",
    "    tokenCounter: new ChatOpenAI({ model: \"gpt-4o\" }),\n",
    "    maxTokens: 80,  // <-- token limit\n",
    "    // The startOn is specified\n",
    "    // to make sure we do not generate a sequence where\n",
    "    // a ToolMessage that contains the result of a tool invocation\n",
    "    // appears before the AIMessage that requested a tool invocation\n",
    "    // as this will cause some chat models to raise an error.\n",
    "    startOn: \"human\",\n",
    "    strategy: \"last\",\n",
    "    includeSystem: true,  // <-- Keep the system message\n",
    "  }\n",
    ")\n",
    "\n",
    "for (const msg of selectedMessages) {\n",
    "    console.log(msg);\n",
    "}"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "0f05d272-2d22-44b7-9fa6-e617a48584b4",
   "metadata": {},
   "source": [
    "```{=mdx}\n",
    "</details>\n",
    "```\n",
    "\n",
    "## Modern usage with LangGraph\n",


# INVALID_PROMPT_INPUT

A [prompt template](/docs/concepts#prompt-templates) received missing or invalid input variables.

One unexpected way this can occur is if you add a JSON object directly into a prompt template:

```ts
import { PromptTemplate } from "@langchain/core/prompts";
import { ChatOpenAI } from "@langchain/openai";

const prompt = PromptTemplate.fromTemplate(`You are a helpful assistant.

Here is an example of how you should respond:

{
  "firstName": "John",
  "lastName": "Doe",
  "age": 21
}

Now, answer the following question:

{question}`);
```

You might think that the above prompt template should require a single input key named `question`, but the JSON object will be
interpreted as an additional variable because the curly braces (`{`) are not escaped, and should be preceded by a second brace instead, like this:

```ts
import { PromptTemplate } from "@langchain/core/prompts";
import { ChatOpenAI } from "@langchain/openai";

const prompt = PromptTemplate.fromTemplate(`You are a helpful assistant.

Here is an example of how you should respond:

{{
  "firstName": "John",
  "lastName": "Doe",
  "age": 21
}}

Now, answer the following question:

{question}`);
```

## Troubleshooting

The following may help resolve this error:

- Double-check your prompt template to ensure that it is correct.
  - If you are using default formatting and you are using curly braces `{` anywhere in your template, they should be double escaped like this: `{{`, as shown above.
- If you are using a [`MessagesPlaceholder`](/docs/concepts/#messagesplaceholder), make sure that you are passing in an array of messages or message-like objects.
  - If you are using shorthand tuples to declare your prompt template, make sure that the variable name is wrapped in curly braces (`["placeholder", "{messages}"]`).
- Try viewing the inputs into your prompt template using [LangSmith](https://docs.smith.langchain.com/) or log statements to confirm they appear as expected.
- If you are pulling a prompt from the [LangChain Prompt Hub](https://smith.langchain.com/prompts), try pulling and logging it or running it in isolation with a sample input to confirm that it is what you expect.


# MESSAGE_COERCION_FAILURE

Several modules in LangChain take [`MessageLike`](https://api.js.langchain.com/types/_langchain_core.messages.BaseMessageLike.html)
objects in place of formal [`BaseMessage`](/docs/concepts#message-types) classes. These include OpenAI style message objects (`{ role: "user", content: "Hello world!" }`),
tuples, and plain strings (which are converted to [`HumanMessages`](/docs/concepts#humanmessage)).

If one of these modules receives a value outside of one of these formats, you will receive an error like the following:

```ts
const badlyFormattedMessageObject = {
  role: "foo",
  randomNonContentValue: "bar",
};

await model.invoke([badlyFormattedMessageObject]);
```

```
Error: Unable to coerce message from array: only human, AI, system, or tool message coercion is currently supported.

Received: {
  "role": "foo",
  "randomNonContentValue": "bar",
}
```

## Troubleshooting

The following may help resolve this error:

- Ensure that all inputs to chat models are an array of LangChain message classes or a supported message-like.
  - Check that there is no stringification or other unexpected transformation occuring.
- Check the error's stack trace and add log or debugger statements.


# OUTPUT_PARSING_FAILURE

An [output parser](/docs/concepts#output-parsers) was unable to handle model output as expected.

To illustrate this, let's say you have an output parser that expects a chat model to output JSON surrounded by a markdown code tag (triple backticks). Here would be an example of good input:

````ts
AIMessage {
  content: "```\n{\"foo\": \"bar\"}\n```"
}
````

Internally, our output parser might try to strip out the markdown fence and newlines and then run `JSON.parse()`.

If instead the chat model generated an output with malformed JSON like this:

````ts
AIMessage {
  content: "```\n{\"foo\":\n```"
}
````

When our output parser attempts to parse this, the `JSON.parse()` call will fail.

Note that some prebuilt constructs like [legacy LangChain agents](/docs/how_to/agent_executor) and chains may use output parsers internally,
so you may see this error even if you're not visibly instantiating and using an output parser.

## Troubleshooting

The following may help resolve this error:

- Consider using [tool calling or other structured output techniques](/docs/how_to/structured_output/) if possible without an output parser to reliably output parseable values.
  - If you are using a prebuilt chain or agent, use [LangGraph](https://langchain-ai.github.io/langgraphjs/) to compose your logic explicitly instead.
- Add more precise formatting instructions to your prompt. In the above example, adding `"You must always return valid JSON fenced by a markdown code block. Do not return any additional text."` to your input may help steer the model to returning the expected format.
- If you are using a smaller or less capable model, try using a more capable one.
- Add [LLM-powered retries](/docs/how_to/output_parser_fixing/).


# MODEL_AUTHENTICATION

Your model provider is denying you access to their service.

## Troubleshooting

The following may help resolve this error:

- Confirm that your API key or other credentials are correct.
- If you are relying on an environment variable to authenticate, confirm that the variable name is correct and that it has a value set.
  - Note that some environments, like Cloudflare Workers, do not support environment variables.
  - For some models, you can try explicitly passing an `apiKey` parameter to rule out any environment variable issues like this:

```ts
const model = new ChatOpenAI({
  apiKey: "YOUR_KEY_HERE",
});
```

- If you are using a proxy or other custom endpoint, make sure that your custom provider does not expect an alternative authentication scheme.


# Few Shot Prompt Templates

Few shot prompting is a prompting technique which provides the Large Language Model (LLM) with a list of examples, and then asks the LLM to generate some text following the lead of the examples provided.

An example of this is the following:

Say you want your LLM to respond in a specific format. You can few shot prompt the LLM with a list of question answer pairs so it knows what format to respond in.

```txt
Respond to the users question in the with the following format:

Question: What is your name?
Answer: My name is John.

Question: What is your age?
Answer: I am 25 years old.

Question: What is your favorite color?
Answer:
```

Here we left the last `Answer:` undefined so the LLM can fill it in. The LLM will then generate the following:

```txt
Answer: I don't have a favorite color; I don't have preferences.
```

### Use Case

In the following example we're few shotting the LLM to rephrase questions into more general queries.

We provide two sets of examples with specific questions, and rephrased general questions. The `FewShotChatMessagePromptTemplate` will use our examples and when `.format` is called, we'll see those examples formatted into a string we can pass to the LLM.

```typescript
import {
  ChatPromptTemplate,
  FewShotChatMessagePromptTemplate,
} from "langchain/prompts";
```

```typescript
const examples = [
  {
    input: "Could the members of The Police perform lawful arrests?",
    output: "what can the members of The Police do?",
  },
  {
    input: "Jan Sindel's was born in what country?",
    output: "what is Jan Sindel's personal history?",
  },
];
const examplePrompt = ChatPromptTemplate.fromTemplate(`Human: {input}
AI: {output}`);
const fewShotPrompt = new FewShotChatMessagePromptTemplate({
  examplePrompt,
  examples,
  inputVariables: [], // no input variables
});
```

```typescript
const formattedPrompt = await fewShotPrompt.format({});
console.log(formattedPrompt);
```

```typescript
[
  HumanMessage {
    lc_namespace: [ 'langchain', 'schema' ],
    content: 'Human: Could the members of The Police perform lawful arrests?\n' +
      'AI: what can the members of The Police do?',
    additional_kwargs: {}
  },
  HumanMessage {
    lc_namespace: [ 'langchain', 'schema' ],
    content: "Human: Jan Sindel's was born in what country?\n" +
      "AI: what is Jan Sindel's personal history?",
    additional_kwargs: {}
  }
]
```

Then, if we use this with another question, the LLM will rephrase the question how we want.

import IntegrationInstallTooltip from "@mdx_components/integration_install_tooltip.mdx";

<IntegrationInstallTooltip></IntegrationInstallTooltip>

```bash npm2yarn
npm install @langchain/openai @langchain/core
```

```typescript
import { ChatOpenAI } from "@langchain/openai";
```

```typescript
const model = new ChatOpenAI({});
const examples = [
  {
    input: "Could the members of The Police perform lawful arrests?",
    output: "what can the members of The Police do?",
  },
  {
    input: "Jan Sindel's was born in what country?",
    output: "what is Jan Sindel's personal history?",
  },
];
const examplePrompt = ChatPromptTemplate.fromTemplate(`Human: {input}
AI: {output}`);
const fewShotPrompt = new FewShotChatMessagePromptTemplate({
  prefix:
    "Rephrase the users query to be more general, using the following examples",
  suffix: "Human: {input}",
  examplePrompt,
  examples,
  inputVariables: ["input"],
});
const formattedPrompt = await fewShotPrompt.format({
  input: "What's France's main city?",
});

const response = await model.invoke(formattedPrompt);
console.log(response);
```

```typescript
AIMessage {
  lc_namespace: [ 'langchain', 'schema' ],
  content: 'What is the capital of France?',
  additional_kwargs: { function_call: undefined }
}
```

### Few Shotting With Functions

You can also partial with a function. The use case for this is when you have a variable you know that you always want to fetch in a common way. A prime example of this is with date or time. Imagine you have a prompt which you always want to have the current date. You can't hard code it in the prompt, and passing it along with the other input variables can be tedious. In this case, it's very handy to be able to partial the prompt with a function that always returns the current date.

```typescript
const getCurrentDate = () => {
  return new Date().toISOString();
};

const prompt = new FewShotChatMessagePromptTemplate({
  template: "Tell me a {adjective} joke about the day {date}",
  inputVariables: ["adjective", "date"],
});

const partialPrompt = await prompt.partial({
  date: getCurrentDate,
});

const formattedPrompt = await partialPrompt.format({
  adjective: "funny",
});

console.log(formattedPrompt);

// Tell me a funny joke about the day 2023-07-13T00:54:59.287Z
```

### Few Shot vs Chat Few Shot

The chat and non chat few shot prompt templates act in a similar way. The below example will demonstrate using chat and non chat, and the differences with their outputs.

```typescript
import {
  FewShotPromptTemplate,
  FewShotChatMessagePromptTemplate,
} from "langchain/prompts";
```

```typescript
const examples = [
  {
    input: "Could the members of The Police perform lawful arrests?",
    output: "what can the members of The Police do?",
  },
  {
    input: "Jan Sindel's was born in what country?",
    output: "what is Jan Sindel's personal history?",
  },
];
const prompt = `Human: {input}
AI: {output}`;
const examplePromptTemplate = PromptTemplate.fromTemplate(prompt);
const exampleChatPromptTemplate = ChatPromptTemplate.fromTemplate(prompt);
const chatFewShotPrompt = new FewShotChatMessagePromptTemplate({
  examplePrompt: exampleChatPromptTemplate,
  examples,
  inputVariables: [], // no input variables
});
const fewShotPrompt = new FewShotPromptTemplate({
  examplePrompt: examplePromptTemplate,
  examples,
  inputVariables: [], // no input variables
});
```

```typescript
console.log("Chat Few Shot: ", await chatFewShotPrompt.formatMessages({}));
/**
Chat Few Shot:  [
  HumanMessage {
    lc_namespace: [ 'langchain', 'schema' ],
    content: 'Human: Could the members of The Police perform lawful arrests?\n' +
      'AI: what can the members of The Police do?',
    additional_kwargs: {}
  },
  HumanMessage {
    lc_namespace: [ 'langchain', 'schema' ],
    content: "Human: Jan Sindel's was born in what country?\n" +
      "AI: what is Jan Sindel's personal history?",
    additional_kwargs: {}
  }
]
 */
```

```typescript
console.log("Few Shot: ", await fewShotPrompt.formatPromptValue({}));
/**
Few Shot:

Human: Could the members of The Police perform lawful arrests?
AI: what can the members of The Police do?

Human: Jan Sindel's was born in what country?
AI: what is Jan Sindel's personal history?
 */
```

Here we can see the main distinctions between `FewShotChatMessagePromptTemplate` and `FewShotPromptTemplate`: input and output values.

`FewShotChatMessagePromptTemplate` works by taking in a list of `ChatPromptTemplate` for examples, and its output is a list of instances of `BaseMessage`.

On the other hand, `FewShotPromptTemplate` works by taking in a `PromptTemplate` for examples, and its output is a string.

## With Non Chat Models

LangChain also provides a class for few shot prompt formatting for non chat models: `FewShotPromptTemplate`. The API is largely the same, but the output is formatted differently (chat messages vs strings).

### Partials With Functions

```typescript
import {
  ChatPromptTemplate,
  FewShotChatMessagePromptTemplate,
} from "langchain/prompts";
```

```typescript
const examplePrompt = PromptTemplate.fromTemplate("{foo}{bar}");
const prompt = new FewShotPromptTemplate({
  prefix: "{foo}{bar}",
  examplePrompt,
  inputVariables: ["foo", "bar"],
});
const partialPrompt = await prompt.partial({
  foo: () => Promise.resolve("boo"),
});
const formatted = await partialPrompt.format({ bar: "baz" });
console.log(formatted);
```

```txt
boobaz\n
```

### With Functions and Example Selector

```typescript
import {
  ChatPromptTemplate,
  FewShotChatMessagePromptTemplate,
} from "langchain/prompts";
```


```typescript
const examplePrompt = PromptTemplate.fromTemplate("An example about {x}");
const exampleSelector = await LengthBasedExampleSelector.fromExamples(
  [{ x: "foo" }, { x: "bar" }],
  { examplePrompt, maxLength: 200 }
);
const prompt = new FewShotPromptTemplate({
  prefix: "{foo}{bar}",
  exampleSelector,
  examplePrompt,
  inputVariables: ["foo", "bar"],
});
const partialPrompt = await prompt.partial({
  foo: () => Promise.resolve("boo"),
});
const formatted = await partialPrompt.format({ bar: "baz" });
console.log(formatted);
```

```txt
boobaz
An example about foo
An example about bar
```


{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# How to do retrieval\n",
    "\n",
    ":::info Prerequisites\n",
    "\n",
    "This guide assumes familiarity with the following:\n",
    "\n",
    "- [Chatbots](/docs/tutorials/chatbot)\n",
    "- [Retrieval-augmented generation](/docs/tutorials/rag)\n",
    "\n",
    ":::\n",
    "\n",
    "Retrieval is a common technique chatbots use to augment their responses with data outside a chat model’s training data. This section will cover how to implement retrieval in the context of chatbots, but it’s worth noting that retrieval is a very subtle and deep topic.\n",
    "\n",
    "## Setup\n",
    "\n",
    "You’ll need to install a few packages, and set any LLM API keys:\n",
    "\n",
    "```{=mdx}\n",
    "import Npm2Yarn from \"@theme/Npm2Yarn\";\n",
    "\n",
    "<Npm2Yarn>\n",
    "  @langchain/openai @langchain/core cheerio\n",
    "</Npm2Yarn>\n",
    "```\n",
    "\n",
    "Let’s also set up a chat model that we’ll use for the below examples.\n",
    "\n",
    "```{=mdx}\n",
    "import ChatModelTabs from \"@theme/ChatModelTabs\";\n",
    "\n",
    "<ChatModelTabs customVarName=\"llm\" />\n",
    "```"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "// @lc-docs-hide-cell\n",
    "\n",
    "import { ChatOpenAI } from \"@langchain/openai\";\n",
    "\n",
    "const llm = new ChatOpenAI({\n",
    "  model: \"gpt-4o\",\n",
    "  temperature: 0,\n",
    "});"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Creating a retriever\n",
    "\n",
    "We’ll use [the LangSmith documentation](https://docs.smith.langchain.com) as source material and store the content in a vectorstore for later retrieval. Note that this example will gloss over some of the specifics around parsing and storing a data source - you can see more [in-depth documentation on creating retrieval systems here](/docs/how_to/#qa-with-rag).\n",
    "\n",
    "Let’s use a document loader to pull text from the docs:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 1,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "\u001b[33m36687\u001b[39m"
      ]
     },
     "execution_count": 1,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "import \"cheerio\";\n",
    "import { CheerioWebBaseLoader } from \"@langchain/community/document_loaders/web/cheerio\";\n",
    "\n",
    "const loader = new CheerioWebBaseLoader(\n",
    "  \"https://docs.smith.langchain.com/user_guide\"\n",
    ");\n",
    "\n",
    "const rawDocs = await loader.load();\n",
    "\n",
    "rawDocs[0].pageContent.length;"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Next, we split it into smaller chunks that the LLM’s context window can handle and store it in a vector database:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 2,
   "metadata": {},
   "outputs": [],
   "source": [
    "import { RecursiveCharacterTextSplitter } from \"langchain/text_splitter\";\n",
    "\n",
    "const textSplitter = new RecursiveCharacterTextSplitter({\n",
    "  chunkSize: 500,\n",
    "  chunkOverlap: 0,\n",
    "});\n",
    "\n",
    "const allSplits = await textSplitter.splitDocuments(rawDocs);"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Then we embed and store those chunks in a vector database:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 3,
   "metadata": {},
   "outputs": [],
   "source": [
    "import { OpenAIEmbeddings } from \"@langchain/openai\";\n",
    "import { MemoryVectorStore } from \"langchain/vectorstores/memory\";\n",
    "\n",
    "const vectorstore = await MemoryVectorStore.fromDocuments(\n",
    "  allSplits,\n",
    "  new OpenAIEmbeddings()\n",
    ");"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "And finally, let’s create a retriever from our initialized vectorstore:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 4,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "[\n",
      "  Document {\n",
      "    pageContent: \"These test cases can be uploaded in bulk, created on the fly, or exported from application traces. L\"... 294 more characters,\n",
      "    metadata: {\n",
      "      source: \"https://docs.smith.langchain.com/user_guide\",\n",
      "      loc: { lines: { from: 7, to: 7 } }\n",
      "    }\n",
      "  },\n",
      "  Document {\n",
      "    pageContent: \"We provide native rendering of chat messages, functions, and retrieve documents.Initial Test SetWhi\"... 347 more characters,\n",
      "    metadata: {\n",
      "      source: \"https://docs.smith.langchain.com/user_guide\",\n",
      "      loc: { lines: { from: 6, to: 6 } }\n",
      "    }\n",
      "  },\n",
      "  Document {\n",
      "    pageContent: \"will help in curation of test cases that can help track regressions/improvements and development of \"... 393 more characters,\n",
      "    metadata: {\n",
      "      source: \"https://docs.smith.langchain.com/user_guide\",\n",
      "      loc: { lines: { from: 11, to: 11 } }\n",
      "    }\n",
      "  },\n",
      "  Document {\n",
      "    pageContent: \"that time period — this is especially handy for debugging production issues.LangSmith also allows fo\"... 396 more characters,\n",
      "    metadata: {\n",
      "      source: \"https://docs.smith.langchain.com/user_guide\",\n",
      "      loc: { lines: { from: 11, to: 11 } }\n",
      "    }\n",
      "  }\n",
      "]\n"
     ]
    }
   ],
   "source": [
    "const retriever = vectorstore.asRetriever(4);\n",
    "\n",
    "const docs = await retriever.invoke(\"how can langsmith help with testing?\");\n",
    "\n",
    "console.log(docs);"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "We can see that invoking the retriever above results in some parts of the LangSmith docs that contain information about testing that our chatbot can use as context when answering questions. And now we’ve got a retriever that can return related data from the LangSmith docs!\n",
    "\n",
    "## Document chains\n",
    "\n",
    "Now that we have a retriever that can return LangChain docs, let’s create a chain that can use them as context to answer questions. We’ll use a `createStuffDocumentsChain` helper function to \"stuff\" all of the input documents into the prompt. It will also handle formatting the docs as strings.\n",
    "\n",
    "In addition to a chat model, the function also expects a prompt that has a `context` variable, as well as a placeholder for chat history messages named `messages`. We’ll create an appropriate prompt and pass it as shown below:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 5,
   "metadata": {},
   "outputs": [],
   "source": [
    "import { createStuffDocumentsChain } from \"langchain/chains/combine_documents\";\n",


{
 "cells": [
  {
   "cell_type": "raw",
   "id": "8165bd4c",
   "metadata": {
    "vscode": {
     "languageId": "raw"
    }
   },
   "source": [
    "---\n",
    "keywords: [memory]\n",
    "---"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "f47033eb",
   "metadata": {},
   "source": [
    "# How to add message history\n",
    "\n",
    ":::info Prerequisites\n",
    "\n",
    "This guide assumes familiarity with the following concepts:\n",
    "\n",
    "- [Chaining runnables](/docs/how_to/sequence/)\n",
    "- [Prompt templates](/docs/concepts/#prompt-templates)\n",
    "- [Chat Messages](/docs/concepts/#message-types)\n",
    "\n",
    ":::\n",
    "\n",
    "```{=mdx}\n",
    ":::note\n",
    "\n",
    "This guide previously covered the [RunnableWithMessageHistory](https://api.js.langchain.com/classes/_langchain_core.runnables.RunnableWithMessageHistory.html) abstraction. You can access this version of the guide in the [v0.2 docs](https://js.langchain.com/v0.2/docs/how_to/message_history/).\n",
    "\n",
    "The LangGraph implementation offers a number of advantages over `RunnableWithMessageHistory`, including the ability to persist arbitrary components of an application's state (instead of only messages).\n",
    "\n",
    ":::\n",
    "```\n",
    "\n",
    "\n",
    "Passing conversation state into and out a chain is vital when building a chatbot. LangGraph implements a built-in persistence layer, allowing chain states to be automatically persisted in memory, or external backends such as SQLite, Postgres or Redis. Details can be found in the LangGraph persistence documentation.\n",
    "\n",
    "In this guide we demonstrate how to add persistence to arbitrary LangChain runnables by wrapping them in a minimal LangGraph application. This lets us persist the message history and other elements of the chain's state, simplifying the development of multi-turn applications. It also supports multiple threads, enabling a single application to interact separately with multiple users.\n",
    "\n",
    "## Setup\n",
    "\n",
    "```{=mdx}\n",
    "import Npm2Yarn from \"@theme/Npm2Yarn\";\n",
    "\n",
    "<Npm2Yarn>\n",
    "  @langchain/core @langchain/langgraph\n",
    "</Npm2Yarn>\n",
    "```\n",
    "\n",
    "Let’s also set up a chat model that we’ll use for the below examples.\n",
    "\n",
    "```{=mdx}\n",
    "import ChatModelTabs from \"@theme/ChatModelTabs\";\n",
    "\n",
    "<ChatModelTabs customVarName=\"llm\" />\n",
    "```\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 30,
   "id": "8a4e4708",
   "metadata": {},
   "outputs": [],
   "source": [
    "// @lc-docs-hide-cell\n",
    "\n",
    "import { ChatOpenAI } from \"@langchain/openai\";\n",
    "\n",
    "const llm = new ChatOpenAI({\n",
    "  model: \"gpt-4o\",\n",
    "  temperature: 0,\n",
    "});"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "1f6121bc-2080-4ccc-acf0-f77de4bc951d",
   "metadata": {},
   "source": [
    "## Example: message inputs\n",
    "\n",
    "Adding memory to a [chat model](/docs/concepts/#chat-models) provides a simple example. Chat models accept a list of messages as input and output a message. LangGraph includes a built-in `MessagesState` that we can use for this purpose.\n",
    "\n",
    "Below, we:\n",
    "1. Define the graph state to be a list of messages;\n",
    "2. Add a single node to the graph that calls a chat model;\n",
    "3. Compile the graph with an in-memory checkpointer to store messages between runs.\n",
    "\n",
    ":::info\n",
    "\n",
    "The output of a LangGraph application is its [state](https://langchain-ai.github.io/langgraphjs/concepts/low_level/).\n",
    "\n",
    ":::"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 31,
   "id": "f691a73a-a866-4354-9fff-8315605e2b8f",
   "metadata": {},
   "outputs": [],
   "source": [
    "import { START, END, MessagesAnnotation, StateGraph, MemorySaver } from \"@langchain/langgraph\";\n",
    "\n",
    "// Define the function that calls the model\n",
    "const callModel = async (state: typeof MessagesAnnotation.State) => {\n",
    "  const response = await llm.invoke(state.messages);\n",
    "  // Update message history with response:\n",
    "  return { messages: response };\n",
    "};\n",
    "\n",
    "// Define a new graph\n",
    "const workflow = new StateGraph(MessagesAnnotation)\n",
    "  // Define the (single) node in the graph\n",
    "  .addNode(\"model\", callModel)\n",
    "  .addEdge(START, \"model\")\n",
    "  .addEdge(\"model\", END);\n",
    "\n",
    "// Add memory\n",
    "const memory = new MemorySaver();\n",
    "const app = workflow.compile({ checkpointer: memory });"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "c0b396a8-f81e-4139-b4b2-75adf61d8179",
   "metadata": {},
   "source": [
    "When we run the application, we pass in a configuration object that specifies a `thread_id`. This ID is used to distinguish conversational threads (e.g., between different users)."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 32,
   "id": "e4309511-2140-4d91-8f5f-ea3661e6d179",
   "metadata": {},
   "outputs": [],
   "source": [
    "import { v4 as uuidv4 } from \"uuid\";\n",
    "\n",
    "const config = { configurable: { thread_id: uuidv4() } }"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "108c45a2-4971-4120-ba64-9a4305a414bb",
   "metadata": {},
   "source": [
    "We can then invoke the application:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 33,
   "id": "72a5ff6c-501f-4151-8dd9-f600f70554be",
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "AIMessage {\n",
      "  \"id\": \"chatcmpl-ABTqCeKnMQmG9IH8dNF5vPjsgXtcM\",\n",
      "  \"content\": \"Hi Bob! How can I assist you today?\",\n",
      "  \"additional_kwargs\": {},\n",
      "  \"response_metadata\": {\n",
      "    \"tokenUsage\": {\n",
      "      \"completionTokens\": 10,\n",
      "      \"promptTokens\": 12,\n",
      "      \"totalTokens\": 22\n",
      "    },\n",
      "    \"finish_reason\": \"stop\",\n",
      "    \"system_fingerprint\": \"fp_e375328146\"\n",
      "  },\n",
      "  \"tool_calls\": [],\n",
      "  \"invalid_tool_calls\": [],\n",
      "  \"usage_metadata\": {\n",
      "    \"input_tokens\": 12,\n",
      "    \"output_tokens\": 10,\n",
      "    \"total_tokens\": 22\n",
      "  }\n",
      "}\n"
     ]
    }
   ],
   "source": [
    "const input = [\n",
    "  {\n",
    "    role: \"user\",\n",
    "    content: \"Hi! I'm Bob.\",\n",
    "  }\n",
    "]\n",
    "const output = await app.invoke({ messages: input }, config)\n",