There is a legacy `agent` concept in LangChain that we are moving towards deprecating: `AgentExecutor`.
AgentExecutor was essentially a runtime for agents.
It was a great place to get started, however, it was not flexible enough as you started to have more customized agents.
In order to solve that we built LangGraph to be this flexible, highly-controllable runtime.

If you are still using AgentExecutor, do not fear: we still have a guide on [how to use AgentExecutor](/docs/how_to/agent_executor).
It is recommended, however, that you start to transition to LangGraph.
In order to assist in this, we have put together a [transition guide on how to do so](/docs/how_to/migrate_agent).

#### ReAct agents

One popular architecture for building agents is [**ReAct**](https://arxiv.org/abs/2210.03629).
ReAct combines reasoning and acting in an iterative process - in fact the name "ReAct" stands for "Reason" and "Act".

The general flow looks like this:

- The model will "think" about what step to take in response to an input and any previous observations.
- The model will then choose an action from available tools (or choose to respond to the user).
- The model will generate arguments to that tool.
- The agent runtime (executor) will parse out the chosen tool and call it with the generated arguments.
- The executor will return the results of the tool call back to the model as an observation.
- This process repeats until the agent chooses to respond.

There are general prompting based implementations that do not require any model-specific features, but the most
reliable implementations use features like [tool calling](/docs/how_to/tool_calling/) to reliably format outputs
and reduce variance.

Please see the [LangGraph documentation](https://langchain-ai.github.io/langgraph/) for more information,
or [this how-to guide](/docs/how_to/migrate_agent/) for specific information on migrating to LangGraph.

### Callbacks

LangChain provides a callbacks system that allows you to hook into the various stages of your LLM application. This is useful for logging, monitoring, streaming, and other tasks.

You can subscribe to these events by using the `callbacks` argument available throughout the API. This argument is list of handler objects, which are expected to implement one or more of the methods described below in more detail.

#### Callback Events

| Event            | Event Trigger                               | Associated Method     |
|------------------|---------------------------------------------|-----------------------|
| Chat model start | When a chat model starts                    | `on_chat_model_start` |
| LLM start        | When a llm starts                           | `on_llm_start`        |
| LLM new token    | When an llm OR chat model emits a new token | `on_llm_new_token`    |
| LLM ends         | When an llm OR chat model ends              | `on_llm_end`          |
| LLM errors       | When an llm OR chat model errors            | `on_llm_error`        |
| Chain start      | When a chain starts running                 | `on_chain_start`      |
| Chain end        | When a chain ends                           | `on_chain_end`        |
| Chain error      | When a chain errors                         | `on_chain_error`      |
| Tool start       | When a tool starts running                  | `on_tool_start`       |
| Tool end         | When a tool ends                            | `on_tool_end`         |
| Tool error       | When a tool errors                          | `on_tool_error`       |
| Agent action     | When an agent takes an action               | `on_agent_action`     |
| Agent finish     | When an agent ends                          | `on_agent_finish`     |
| Retriever start  | When a retriever starts                     | `on_retriever_start`  |
| Retriever end    | When a retriever ends                       | `on_retriever_end`    |
| Retriever error  | When a retriever errors                     | `on_retriever_error`  |
| Text             | When arbitrary text is run                  | `on_text`             |
| Retry            | When a retry event is run                   | `on_retry`            |

#### Callback handlers

Callback handlers can either be `sync` or `async`:

* Sync callback handlers implement the [BaseCallbackHandler](https://python.langchain.com/api_reference/core/callbacks/langchain_core.callbacks.base.BaseCallbackHandler.html) interface.
* Async callback handlers implement the [AsyncCallbackHandler](https://python.langchain.com/api_reference/core/callbacks/langchain_core.callbacks.base.AsyncCallbackHandler.html) interface.

During run-time LangChain configures an appropriate callback manager (e.g., [CallbackManager](https://python.langchain.com/api_reference/core/callbacks/langchain_core.callbacks.manager.CallbackManager.html) or [AsyncCallbackManager](https://python.langchain.com/api_reference/core/callbacks/langchain_core.callbacks.manager.AsyncCallbackManager.html)) which will be responsible for calling the appropriate method on each "registered" callback handler when the event is triggered.

#### Passing callbacks

The `callbacks` property is available on most objects throughout the API (Models, Tools, Agents, etc.) in two different places:

- **Request time callbacks**: Passed at the time of the request in addition to the input data.
    Available on all standard `Runnable` objects. These callbacks are INHERITED by all children
    of the object they are defined on. For example, `chain.invoke({"number": 25}, {"callbacks": [handler]})`.
- **Constructor callbacks**: `chain = TheNameOfSomeChain(callbacks=[handler])`. These callbacks
    are passed as arguments to the constructor of the object. The callbacks are scoped
    only to the object they are defined on, and are **not** inherited by any children of the object.

:::warning
Constructor callbacks are scoped only to the object they are defined on. They are **not** inherited by children
of the object.
:::

If you're creating a custom chain or runnable, you need to remember to propagate request time
callbacks to any child objects.

:::important Async in Python<=3.10

Any `RunnableLambda`, a `RunnableGenerator`, or `Tool` that invokes other runnables
and is running `async` in python<=3.10, will have to propagate callbacks to child
objects manually. This is because LangChain cannot automatically propagate
callbacks to child objects in this case.

This is a common reason why you may fail to see events being emitted from custom
runnables or tools.
:::

For specifics on how to use callbacks, see the [relevant how-to guides here](/docs/how_to/#callbacks).

## Techniques

### Streaming

Individual LLM calls often run for much longer than traditional resource requests.
This compounds when you build more complex chains or agents that require multiple reasoning steps.

Fortunately, LLMs generate output iteratively, which means it's possible to show sensible intermediate results
before the final response is ready. Consuming output as soon as it becomes available has therefore become a vital part of the UX
around building apps with LLMs to help alleviate latency issues, and LangChain aims to have first-class support for streaming.

Below, we'll discuss some concepts and considerations around streaming in LangChain.

#### `.stream()` and `.astream()`

Most modules in LangChain include the `.stream()` method (and the equivalent `.astream()` method for [async](https://docs.python.org/3/library/asyncio.html) environments) as an ergonomic streaming interface.
`.stream()` returns an iterator, which you can consume with a simple `for` loop. Here's an example with a chat model:

```python
from langchain_anthropic import ChatAnthropic

model = ChatAnthropic(model="claude-3-sonnet-20240229")

for chunk in model.stream("what color is the sky?"):
    print(chunk.content, end="|", flush=True)
```

For models (or other components) that don't support streaming natively, this iterator would just yield a single chunk, but
you could still use the same general pattern when calling them. Using `.stream()` will also automatically call the model in streaming mode
without the need to provide additional config.

The type of each outputted chunk depends on the type of component - for example, chat models yield [`AIMessageChunks`](https://python.langchain.com/api_reference/core/messages/langchain_core.messages.ai.AIMessageChunk.html).
Because this method is part of [LangChain Expression Language](/docs/concepts/#langchain-expression-language-lcel),
you can handle formatting differences from different outputs using an [output parser](/docs/concepts/#output-parsers) to transform
each yielded chunk.

You can check out [this guide](/docs/how_to/streaming/#using-stream) for more detail on how to use `.stream()`.

#### `.astream_events()`

While the `.stream()` method is intuitive, it can only return the final generated value of your chain. This is fine for single LLM calls,
but as you build more complex chains of several LLM calls together, you may want to use the intermediate values of
the chain alongside the final output - for example, returning sources alongside the final generation when building a chat
over documents app.


There are ways to do this [using callbacks](/docs/concepts/#callbacks-1), or by constructing your chain in such a way that it passes intermediate
values to the end with something like chained [`.assign()`](/docs/how_to/passthrough/) calls, but LangChain also includes an
`.astream_events()` method that combines the flexibility of callbacks with the ergonomics of `.stream()`. When called, it returns an iterator
which yields [various types of events](/docs/how_to/streaming/#event-reference) that you can filter and process according
to the needs of your project.

Here's one small example that prints just events containing streamed chat model output:

```python
from langchain_core.output_parsers import StrOutputParser
from langchain_core.prompts import ChatPromptTemplate
from langchain_anthropic import ChatAnthropic

model = ChatAnthropic(model="claude-3-sonnet-20240229")

prompt = ChatPromptTemplate.from_template("tell me a joke about {topic}")
parser = StrOutputParser()
chain = prompt | model | parser

async for event in chain.astream_events({"topic": "parrot"}, version="v2"):
    kind = event["event"]
    if kind == "on_chat_model_stream":
        print(event, end="|", flush=True)
```

You can roughly think of it as an iterator over callback events (though the format differs) - and you can use it on almost all LangChain components!

See [this guide](/docs/how_to/streaming/#using-stream-events) for more detailed information on how to use `.astream_events()`,
including a table listing available events.

#### Callbacks

The lowest level way to stream outputs from LLMs in LangChain is via the [callbacks](/docs/concepts/#callbacks) system. You can pass a
callback handler that handles the [`on_llm_new_token`](https://python.langchain.com/api_reference/langchain/callbacks/langchain.callbacks.streaming_aiter.AsyncIteratorCallbackHandler.html#langchain.callbacks.streaming_aiter.AsyncIteratorCallbackHandler.on_llm_new_token) event into LangChain components. When that component is invoked, any
[LLM](/docs/concepts/#llms) or [chat model](/docs/concepts/#chat-models) contained in the component calls
the callback with the generated token. Within the callback, you could pipe the tokens into some other destination, e.g. a HTTP response.
You can also handle the [`on_llm_end`](https://python.langchain.com/api_reference/langchain/callbacks/langchain.callbacks.streaming_aiter.AsyncIteratorCallbackHandler.html#langchain.callbacks.streaming_aiter.AsyncIteratorCallbackHandler.on_llm_end) event to perform any necessary cleanup.

You can see [this how-to section](/docs/how_to/#callbacks) for more specifics on using callbacks.

Callbacks were the first technique for streaming introduced in LangChain. While powerful and generalizable,
they can be unwieldy for developers. For example:

- You need to explicitly initialize and manage some aggregator or other stream to collect results.
- The execution order isn't explicitly guaranteed, and you could theoretically have a callback run after the `.invoke()` method finishes.
- Providers would often make you pass an additional parameter to stream outputs instead of returning them all at once.
- You would often ignore the result of the actual model call in favor of callback results.

#### Tokens

The unit that most model providers use to measure input and output is via a unit called a **token**.
Tokens are the basic units that language models read and generate when processing or producing text.
The exact definition of a token can vary depending on the specific way the model was trained -
for instance, in English, a token could be a single word like "apple", or a part of a word like "app".

When you send a model a prompt, the words and characters in the prompt are encoded into tokens using a **tokenizer**.
The model then streams back generated output tokens, which the tokenizer decodes into human-readable text.
The below example shows how OpenAI models tokenize `LangChain is cool!`:

![](/img/tokenization.png)

You can see that it gets split into 5 different tokens, and that the boundaries between tokens are not exactly the same as word boundaries.

The reason language models use tokens rather than something more immediately intuitive like "characters"
has to do with how they process and understand text. At a high-level, language models iteratively predict their next generated output based on
the initial input and their previous generations. Training the model using tokens language models to handle linguistic
units (like words or subwords) that carry meaning, rather than individual characters, which makes it easier for the model
to learn and understand the structure of the language, including grammar and context.
Furthermore, using tokens can also improve efficiency, since the model processes fewer units of text compared to character-level processing.

### Function/tool calling

:::info
We use the term `tool calling` interchangeably with `function calling`. Although
function calling is sometimes meant to refer to invocations of a single function,
we treat all models as though they can return multiple tool or function calls in
each message.
:::

Tool calling allows a [chat model](/docs/concepts/#chat-models) to respond to a given prompt by generating output that
matches a user-defined schema.

While the name implies that the model is performing
some action, this is actually not the case! The model only generates the arguments to a tool, and actually running the tool (or not) is up to the user.
One common example where you **wouldn't** want to call a function with the generated arguments
is if you want to [extract structured output matching some schema](/docs/concepts/#structured-output)
from unstructured text. You would give the model an "extraction" tool that takes
parameters matching the desired schema, then treat the generated output as your final
result.

![Diagram of a tool call by a chat model](/img/tool_call.png)

Tool calling is not universal, but is supported by many popular LLM providers, including [Anthropic](/docs/integrations/chat/anthropic/),
[Cohere](/docs/integrations/chat/cohere/), [Google](/docs/integrations/chat/google_vertex_ai_palm/),
[Mistral](/docs/integrations/chat/mistralai/), [OpenAI](/docs/integrations/chat/openai/), and even for locally-running models via [Ollama](/docs/integrations/chat/ollama/).

LangChain provides a standardized interface for tool calling that is consistent across different models.

The standard interface consists of:

* `ChatModel.bind_tools()`: a method for specifying which tools are available for a model to call. This method accepts [LangChain tools](/docs/concepts/#tools) as well as [Pydantic](https://pydantic.dev/) objects.
* `AIMessage.tool_calls`: an attribute on the `AIMessage` returned from the model for accessing the tool calls requested by the model.

#### Tool usage

After the model calls tools, you can use the tool by invoking it, then passing the arguments back to the model.
LangChain provides the [`Tool`](/docs/concepts/#tools) abstraction to help you handle this.

The general flow is this:

1. Generate tool calls with a chat model in response to a query.
2. Invoke the appropriate tools using the generated tool call as arguments.
3. Format the result of the tool invocations as [`ToolMessages`](/docs/concepts/#toolmessage).
4. Pass the entire list of messages back to the model so that it can generate a final answer (or call more tools).

![Diagram of a complete tool calling flow](/img/tool_calling_flow.png)

This is how tool calling [agents](/docs/concepts/#agents) perform tasks and answer queries.

Check out some more focused guides below:

- [How to use chat models to call tools](/docs/how_to/tool_calling/)
- [How to pass tool outputs to chat models](/docs/how_to/tool_results_pass_to_model/)
- [Building an agent with LangGraph](https://langchain-ai.github.io/langgraph/tutorials/introduction/)

### Structured output

LLMs are capable of generating arbitrary text. This enables the model to respond appropriately to a wide
range of inputs, but for some use-cases, it can be useful to constrain the LLM's output
to a specific format or structure. This is referred to as **structured output**.

For example, if the output is to be stored in a relational database,
it is much easier if the model generates output that adheres to a defined schema or format.
[Extracting specific information](/docs/tutorials/extraction/) from unstructured text is another
case where this is particularly useful. Most commonly, the output format will be JSON,
though other formats such as [YAML](/docs/how_to/output_parser_yaml/) can be useful too. Below, we'll discuss
a few ways to get structured output from models in LangChain.

#### `.with_structured_output()`

For convenience, some LangChain chat models support a [`.with_structured_output()`](/docs/how_to/structured_output/#the-with_structured_output-method)
method. This method only requires a schema as input, and returns a dict or Pydantic object.
Generally, this method is only present on models that support one of the more advanced methods described below,
and will use one of them under the hood. It takes care of importing a suitable output parser and
formatting the schema in the right format for the model.

Here's an example:

```python
from typing import Optional

from pydantic import BaseModel, Field


class Joke(BaseModel):
    """Joke to tell user."""

    setup: str = Field(description="The setup of the joke")
    punchline: str = Field(description="The punchline to the joke")
    rating: Optional[int] = Field(description="How funny the joke is, from 1 to 10")

structured_llm = llm.with_structured_output(Joke)

structured_llm.invoke("Tell me a joke about cats")
```

```
Joke(setup='Why was the cat sitting on the computer?', punchline='To keep an eye on the mouse!', rating=None)
```

We recommend this method as a starting point when working with structured output:

- It uses other model-specific features under the hood, without the need to import an output parser.
- For the models that use tool calling, no special prompting is needed.
- If multiple underlying techniques are supported, you can supply a `method` parameter to
[toggle which one is used](/docs/how_to/structured_output/#advanced-specifying-the-method-for-structuring-outputs).

You may want or need to use other techniques if:

- The chat model you are using does not support tool calling.
- You are working with very complex schemas and the model is having trouble generating outputs that conform.

For more information, check out this [how-to guide](/docs/how_to/structured_output/#the-with_structured_output-method).

You can also check out [this table](/docs/integrations/chat/#advanced-features) for a list of models that support
`with_structured_output()`.

#### Raw prompting

The most intuitive way to get a model to structure output is to ask nicely.
In addition to your query, you can give instructions describing what kind of output you'd like, then
parse the output using an [output parser](/docs/concepts/#output-parsers) to convert the raw
model message or string output into something more easily manipulated.

The biggest benefit to raw prompting is its flexibility:

- Raw prompting does not require any special model features, only sufficient reasoning capability to understand
the passed schema.
- You can prompt for any format you'd like, not just JSON. This can be useful if the model you
are using is more heavily trained on a certain type of data, such as XML or YAML.

However, there are some drawbacks too:

- LLMs are non-deterministic, and prompting a LLM to consistently output data in the exactly correct format
for smooth parsing can be surprisingly difficult and model-specific.
- Individual models have quirks depending on the data they were trained on, and optimizing prompts can be quite difficult.
Some may be better at interpreting [JSON schema](https://json-schema.org/), others may be best with TypeScript definitions,
and still others may prefer XML.

While features offered by model providers may increase reliability, prompting techniques remain important for tuning your
results no matter which method you choose.

#### JSON mode
<span data-heading-keywords="json mode"></span>

Some models, such as [Mistral](/docs/integrations/chat/mistralai/), [OpenAI](/docs/integrations/chat/openai/),
[Together AI](/docs/integrations/chat/together/) and [Ollama](/docs/integrations/chat/ollama/),
support a feature called **JSON mode**, usually enabled via config.

When enabled, JSON mode will constrain the model's output to always be some sort of valid JSON.
Often they require some custom prompting, but it's usually much less burdensome than completely raw prompting and
more along the lines of, `"you must always return JSON"`. The [output also generally easier to parse](/docs/how_to/output_parser_json/).

It's also generally simpler to use directly and more commonly available than tool calling, and can give
more flexibility around prompting and shaping results than tool calling.

Here's an example:

```python
from langchain_core.prompts import ChatPromptTemplate
from langchain_openai import ChatOpenAI
from langchain.output_parsers.json import SimpleJsonOutputParser

model = ChatOpenAI(
    model="gpt-4o",
    model_kwargs={ "response_format": { "type": "json_object" } },
)

prompt = ChatPromptTemplate.from_template(
    "Answer the user's question to the best of your ability."
    'You must always output a JSON object with an "answer" key and a "followup_question" key.'
    "{question}"
)

chain = prompt | model | SimpleJsonOutputParser()

chain.invoke({ "question": "What is the powerhouse of the cell?" })
```

```
{'answer': 'The powerhouse of the cell is the mitochondrion. It is responsible for producing energy in the form of ATP through cellular respiration.',
 'followup_question': 'Would you like to know more about how mitochondria produce energy?'}
```

For a full list of model providers that support JSON mode, see [this table](/docs/integrations/chat/#advanced-features).

#### Tool calling {#structured-output-tool-calling}

For models that support it, [tool calling](/docs/concepts/#functiontool-calling) can be very convenient for structured output. It removes the
guesswork around how best to prompt schemas in favor of a built-in model feature.

It works by first binding the desired schema either directly or via a [LangChain tool](/docs/concepts/#tools) to a
[chat model](/docs/concepts/#chat-models) using the `.bind_tools()` method. The model will then generate an `AIMessage` containing
a `tool_calls` field containing `args` that match the desired shape.

There are several acceptable formats you can use to bind tools to a model in LangChain. Here's one example:

```python
from pydantic import BaseModel, Field
from langchain_openai import ChatOpenAI

class ResponseFormatter(BaseModel):
    """Always use this tool to structure your response to the user."""

    answer: str = Field(description="The answer to the user's question")
    followup_question: str = Field(description="A followup question the user could ask")

model = ChatOpenAI(
    model="gpt-4o",
    temperature=0,
)

model_with_tools = model.bind_tools([ResponseFormatter])

ai_msg = model_with_tools.invoke("What is the powerhouse of the cell?")

ai_msg.tool_calls[0]["args"]
```

```
{'answer': "The powerhouse of the cell is the mitochondrion. It generates most of the cell's supply of adenosine triphosphate (ATP), which is used as a source of chemical energy.",
 'followup_question': 'How do mitochondria generate ATP?'}
```

Tool calling is a generally consistent way to get a model to generate structured output, and is the default technique
used for the [`.with_structured_output()`](/docs/concepts/#with_structured_output) method when a model supports it.

The following how-to guides are good practical resources for using function/tool calling for structured output:

- [How to return structured data from an LLM](/docs/how_to/structured_output/)
- [How to use a model to call tools](/docs/how_to/tool_calling)

For a full list of model providers that support tool calling, [see this table](/docs/integrations/chat/#advanced-features).

### Few-shot prompting

One of the most effective ways to improve model performance is to give a model examples of
what you want it to do. The technique of adding example inputs and expected outputs
to a model prompt is known as "few-shot prompting". The technique is based on the
[Language Models are Few-Shot Learners](https://arxiv.org/abs/2005.14165) paper.
There are a few things to think about when doing few-shot prompting:

1. How are examples generated?
2. How many examples are in each prompt?
3. How are examples selected at runtime?
4. How are examples formatted in the prompt?

Here are the considerations for each.

#### 1. Generating examples

The first and most important step of few-shot prompting is coming up with a good dataset of examples. Good examples should be relevant at runtime, clear, informative, and provide information that was not already known to the model.

At a high-level, the basic ways to generate examples are:
- Manual: a person/people generates examples they think are useful.
- Better model: a better (presumably more expensive/slower) model's responses are used as examples for a worse (presumably cheaper/faster) model.
- User feedback: users (or labelers) leave feedback on interactions with the application and examples are generated based on that feedback (for example, all interactions with positive feedback could be turned into examples).
- LLM feedback: same as user feedback but the process is automated by having models evaluate themselves.

Which approach is best depends on your task. For tasks where a small number core principles need to be understood really well, it can be valuable hand-craft a few really good examples.
For tasks where the space of correct behaviors is broader and more nuanced, it can be useful to generate many examples in a more automated fashion so that there's a higher likelihood of there being some highly relevant examples for any runtime input.

**Single-turn v.s. multi-turn examples**

Another dimension to think about when generating examples is what the example is actually showing.

The simplest types of examples just have a user input and an expected model output. These are single-turn examples.


One more complex type if example is where the example is an entire conversation, usually in which a model initially responds incorrectly and a user then tells the model how to correct its answer.
This is called a multi-turn example. Multi-turn examples can be useful for more nuanced tasks where its useful to show common errors and spell out exactly why they're wrong and what should be done instead.

#### 2. Number of examples

Once we have a dataset of examples, we need to think about how many examples should be in each prompt.
The key tradeoff is that more examples generally improve performance, but larger prompts increase costs and latency.
And beyond some threshold having too many examples can start to confuse the model.
Finding the right number of examples is highly dependent on the model, the task, the quality of the examples, and your cost and latency constraints.
Anecdotally, the better the model is the fewer examples it needs to perform well and the more quickly you hit steeply diminishing returns on adding more examples.
But, the best/only way to reliably answer this question is to run some experiments with different numbers of examples.

#### 3. Selecting examples

Assuming we are not adding our entire example dataset into each prompt, we need to have a way of selecting examples from our dataset based on a given input. We can do this:
- Randomly
- By (semantic or keyword-based) similarity of the inputs
- Based on some other constraints, like token size

LangChain has a number of [`ExampleSelectors`](/docs/concepts/#example-selectors) which make it easy to use any of these techniques.

Generally, selecting by semantic similarity leads to the best model performance. But how important this is is again model and task specific, and is something worth experimenting with.

#### 4. Formatting examples

Most state-of-the-art models these days are chat models, so we'll focus on formatting examples for those. Our basic options are to insert the examples:
- In the system prompt as a string
- As their own messages

If we insert our examples into the system prompt as a string, we'll need to make sure it's clear to the model where each example begins and which parts are the input versus output. Different models respond better to different syntaxes, like [ChatML](https://learn.microsoft.com/en-us/azure/ai-services/openai/how-to/chat-markup-language), XML, TypeScript, etc.

If we insert our examples as messages, where each example is represented as a sequence of Human, AI messages, we might want to also assign [names](/docs/concepts/#messages) to our messages like `"example_user"` and `"example_assistant"` to make it clear that these messages correspond to different actors than the latest input message.

**Formatting tool call examples**

One area where formatting examples as messages can be tricky is when our example outputs have tool calls. This is because different models have different constraints on what types of message sequences are allowed when any tool calls are generated.
- Some models require that any AIMessage with tool calls be immediately followed by ToolMessages for every tool call,
- Some models additionally require that any ToolMessages be immediately followed by an AIMessage before the next HumanMessage,
- Some models require that tools are passed in to the model if there are any tool calls / ToolMessages in the chat history.

These requirements are model-specific and should be checked for the model you are using. If your model requires ToolMessages after tool calls and/or AIMessages after ToolMessages and your examples only include expected tool calls and not the actual tool outputs, you can try adding dummy ToolMessages / AIMessages to the end of each example with generic contents to satisfy the API constraints.
In these cases it's especially worth experimenting with inserting your examples as strings versus messages, as having dummy messages can adversely affect certain models.

You can see a case study of how Anthropic and OpenAI respond to different few-shot prompting techniques on two different tool calling benchmarks [here](https://blog.langchain.dev/few-shot-prompting-to-improve-tool-calling-performance/).

### Retrieval

LLMs are trained on a large but fixed dataset, limiting their ability to reason over private or recent information.
Fine-tuning an LLM with specific facts is one way to mitigate this, but is often [poorly suited for factual recall](https://www.anyscale.com/blog/fine-tuning-is-for-form-not-facts) and [can be costly](https://www.glean.com/blog/how-to-build-an-ai-assistant-for-the-enterprise).
`Retrieval` is the process of providing relevant information to an LLM to improve its response for a given input.
`Retrieval augmented generation` (`RAG`) [paper](https://arxiv.org/abs/2005.11401) is the process of grounding the LLM generation (output) using the retrieved information.

:::tip

* See our RAG from Scratch [code](https://github.com/langchain-ai/rag-from-scratch) and [video series](https://youtube.com/playlist?list=PLfaIDFEXuae2LXbO1_PKyVJiQ23ZztA0x&feature=shared).
* For a high-level guide on retrieval, see this [tutorial on RAG](/docs/tutorials/rag/).

:::

RAG is only as good as the retrieved documents’ relevance and quality. Fortunately, an emerging set of techniques can be employed to design and improve RAG systems. We've focused on taxonomizing and summarizing many of these techniques (see below figure) and will share some high-level strategic guidance in the following sections.
You can and should experiment with using different pieces together. You might also find [this LangSmith guide](https://docs.smith.langchain.com/how_to_guides/evaluation/evaluate_llm_application) useful for showing how to evaluate different iterations of your app.

![](/img/rag_landscape.png)

#### Query Translation

First, consider the user input(s) to your RAG system. Ideally, a RAG system can handle a wide range of inputs, from poorly worded questions to complex multi-part queries.
**Using an LLM to review and optionally modify the input is the central idea behind query translation.** This serves as a general buffer, optimizing raw user inputs for your retrieval system.
For example, this can be as simple as extracting keywords or as complex as generating multiple sub-questions for a complex query.

| Name          | When to use | Description                                                                                |
|---------------|-------------|----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------|
| [Multi-query](/docs/how_to/MultiQueryRetriever/)   | When you need to cover multiple perspectives of a question. | Rewrite the user question from multiple perspectives, retrieve documents for each rewritten question, return the unique documents for all queries. |
| [Decomposition](https://github.com/langchain-ai/rag-from-scratch/blob/main/rag_from_scratch_5_to_9.ipynb) | When a question can be broken down into smaller subproblems. | Decompose a question into a set of subproblems / questions, which can either be solved sequentially (use the answer from first + retrieval to answer the second) or in parallel (consolidate each answer into final answer).                                                           |
| [Step-back](https://github.com/langchain-ai/rag-from-scratch/blob/main/rag_from_scratch_5_to_9.ipynb)     | When a higher-level conceptual understanding is required. | First prompt the LLM to ask a generic step-back question about higher-level concepts or principles, and retrieve relevant facts about them. Use this grounding to help answer the user question. [Paper](https://arxiv.org/pdf/2310.06117).                                            |
| [HyDE](https://github.com/langchain-ai/rag-from-scratch/blob/main/rag_from_scratch_5_to_9.ipynb)          | If you have challenges retrieving relevant documents using the raw user inputs. | Use an LLM to convert questions into hypothetical documents that answer the question. Use the embedded hypothetical documents to retrieve real documents with the premise that doc-doc similarity search can produce more relevant matches. [Paper](https://arxiv.org/abs/2212.10496). |

:::tip

See our RAG from Scratch videos for a few different specific approaches:
- [Multi-query](https://youtu.be/JChPi0CRnDY?feature=shared)
- [Decomposition](https://youtu.be/h0OPWlEOank?feature=shared)
- [Step-back](https://youtu.be/xn1jEjRyJ2U?feature=shared)
- [HyDE](https://youtu.be/SaDzIVkYqyY?feature=shared)

:::

#### Routing

Second, consider the data sources available to your RAG system. You want to query across more than one database or across structured and unstructured data sources. **Using an LLM to review the input and route it to the appropriate data source is a simple and effective approach for querying across sources.**

| Name             | When to use                                | Description |
|------------------|--------------------------------------------|-------------|
| [Logical routing](/docs/how_to/routing/)  | When you can prompt an LLM with rules to decide where to route the input. | Logical routing can use an LLM to reason about the query and choose which datastore is most appropriate. |
| [Semantic routing](/docs/how_to/routing/#routing-by-semantic-similarity) | When semantic similarity is an effective way to determine where to route the input. | Semantic routing embeds both query and, typically a set of prompts. It then chooses the appropriate prompt based upon similarity. |

:::tip


| Name                      | Index Type                   | Uses an LLM               | When to Use                                                                                | Description                                                                                |
|---------------------------|------------------------------|---------------------------|-----------------------------------------------------------------------------------------------------------------------------------------------|--------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------|
| [Contextual Compression](/docs/how_to/contextual_compression/)    | Any                          | Sometimes                 | If you are finding that your retrieved documents contain too much irrelevant information and are distracting the LLM.                         | This puts a post-processing step on top of another retriever and extracts only the most relevant information from retrieved documents. This can be done with embeddings or an LLM.                                                                                |
| [Ensemble](/docs/how_to/ensemble_retriever/)                  | Any                          | No                        | If you have multiple retrieval methods and want to try combining them.                                                                        | This fetches documents from multiple retrievers and then combines them.                                                                                |
| [Re-ranking](/docs/integrations/retrievers/cohere-reranker/)                  | Any                          | Yes                        |  If you want to rank retrieved documents based upon relevance, especially if you want to combine results from multiple retrieval methods .                                                                         | Given a query and a list of documents, Rerank indexes the documents from most to least semantically relevant to the query.                                                                                |

:::tip

See our RAG from Scratch video on [RAG-Fusion](https://youtu.be/77qELPbNgxA?feature=shared) ([paper](https://arxiv.org/abs/2402.03367)), on approach for post-processing across multiple queries:  Rewrite the user question from multiple perspectives, retrieve documents for each rewritten question, and combine the ranks of multiple search result lists to produce a single, unified ranking with [Reciprocal Rank Fusion (RRF)](https://towardsdatascience.com/forget-rag-the-future-is-rag-fusion-1147298d8ad1).

:::

#### Generation

**Finally, consider ways to build self-correction into your RAG system.** RAG systems can suffer from low quality retrieval (e.g., if a user question is out of the domain for the index) and / or hallucinations in generation. A naive retrieve-generate pipeline has no ability to detect or self-correct from these kinds of errors. The concept of ["flow engineering"](https://x.com/karpathy/status/1748043513156272416) has been introduced [in the context of code generation](https://arxiv.org/abs/2401.08500): iteratively build an answer to a code question with unit tests to check and self-correct errors. Several works have applied this RAG, such as Self-RAG and Corrective-RAG. In both cases, checks for document relevance, hallucinations, and / or answer quality are performed in the RAG answer generation flow.

We've found that graphs are a great way to reliably express logical flows and have implemented ideas from several of these papers [using LangGraph](https://github.com/langchain-ai/langgraph/tree/main/examples/rag), as shown in the figure below (red - routing, blue - fallback, green - self-correction):
- **Routing:** Adaptive RAG ([paper](https://arxiv.org/abs/2403.14403)). Route questions to different retrieval approaches, as discussed above
- **Fallback:** Corrective RAG ([paper](https://arxiv.org/pdf/2401.15884.pdf)). Fallback to web search if docs are not relevant to query
- **Self-correction:** Self-RAG ([paper](https://arxiv.org/abs/2310.11511)). Fix answers w/ hallucinations or don’t address question

![](/img/langgraph_rag.png)

| Name              | When to use                                               | Description |
|-------------------|-----------------------------------------------------------|-------------|
| Self-RAG          | When needing to fix answers with hallucinations or irrelevant content. | Self-RAG performs checks for document relevance, hallucinations, and answer quality during the RAG answer generation flow, iteratively building an answer and self-correcting errors. |
| Corrective-RAG    | When needing a fallback mechanism for low relevance docs. | Corrective-RAG includes a fallback (e.g., to web search) if the retrieved documents are not relevant to the query, ensuring higher quality and more relevant retrieval. |

:::tip

See several videos and cookbooks showcasing RAG with LangGraph:
- [LangGraph Corrective RAG](https://www.youtube.com/watch?v=E2shqsYwxck)
- [LangGraph combining Adaptive, Self-RAG, and Corrective RAG](https://www.youtube.com/watch?v=-ROS6gfYIts)
- [Cookbooks for RAG using LangGraph](https://github.com/langchain-ai/langgraph/tree/main/examples/rag)

See our LangGraph RAG recipes with partners:
- [Meta](https://github.com/meta-llama/llama-recipes/tree/main/recipes/3p_integrations/langchain)
- [Mistral](https://github.com/mistralai/cookbook/tree/main/third_party/langchain)

:::

### Text splitting

LangChain offers many different types of `text splitters`.
These all live in the `langchain-text-splitters` package.

Table columns:

- **Name**: Name of the text splitter
- **Classes**: Classes that implement this text splitter
- **Splits On**: How this text splitter splits text
- **Adds Metadata**: Whether or not this text splitter adds metadata about where each chunk came from.
- **Description**: Description of the splitter, including recommendation on when to use it.


# langchain

## 0.2.0

### Deleted

As of release 0.2.0, `langchain` is required to be integration-agnostic. This means that code in `langchain`  should not by default instantiate any specific chat models, llms, embedding models, vectorstores etc; instead, the user will be required to specify those explicitly.

The following functions and classes require an explicit LLM to be passed as an argument:

- `langchain.agents.agent_toolkits.vectorstore.toolkit.VectorStoreToolkit`
- `langchain.agents.agent_toolkits.vectorstore.toolkit.VectorStoreRouterToolkit`
- `langchain.chains.openai_functions.get_openapi_chain`
- `langchain.chains.router.MultiRetrievalQAChain.from_retrievers`
- `langchain.indexes.VectorStoreIndexWrapper.query`
- `langchain.indexes.VectorStoreIndexWrapper.query_with_sources`
- `langchain.indexes.VectorStoreIndexWrapper.aquery_with_sources`
- `langchain.chains.flare.FlareChain`

The following classes now require passing an explicit Embedding model as an argument:

- `langchain.indexes.VectostoreIndexCreator`

The following code has been removed:

- `langchain.natbot.NatBotChain.from_default` removed in favor of the `from_llm` class method.

### Deprecated

We have two main types of deprecations:

1. Code that was moved from `langchain` into another package (e.g, `langchain-community`)

If you try to import it from `langchain`, the import will keep on working, but will raise a deprecation warning. The warning will provide a replacement import statement.

```python
python -c "from langchain.document_loaders.markdown import UnstructuredMarkdownLoader"

```

```python
LangChainDeprecationWarning: Importing UnstructuredMarkdownLoader from langchain.document_loaders is deprecated. Please replace deprecated imports:

>> from langchain.document_loaders import UnstructuredMarkdownLoader

with new imports of:

>> from langchain_community.document_loaders import UnstructuredMarkdownLoader
```

We will continue supporting the imports in `langchain` until release 0.4 as long as the relevant package where the code lives is installed. (e.g., as long as `langchain_community` is installed.)

However, we advise for users to not rely on these imports and instead migrate to the new imports. To help with this process, we’re releasing a migration script via the LangChain CLI. See further instructions in migration guide.

1. Code that has better alternatives available and will eventually be removed, so there’s only a single way to do things. (e.g., `predict_messages` method in ChatModels has been deprecated in favor of `invoke`).

Many of these were marked for removal in 0.2. We have bumped the removal to 0.3.

## 0.1.0 (Jan 5, 2024)

### Deleted

No deletions.

### Deprecated

Deprecated classes and methods will be removed in 0.2.0

| Deprecated                      | Alternative                       | Reason                                         |
|---------------------------------|-----------------------------------|------------------------------------------------|
| ChatVectorDBChain               | ConversationalRetrievalChain      | More general to all retrievers                 |
| create_ernie_fn_chain           | create_ernie_fn_runnable          | Use LCEL under the hood                        |
| created_structured_output_chain | create_structured_output_runnable | Use LCEL under the hood                        |
| NatBotChain                     |                                   | Not used                                       |
| create_openai_fn_chain          | create_openai_fn_runnable         | Use LCEL under the hood                        |
| create_structured_output_chain  | create_structured_output_runnable | Use LCEL under the hood                        |
| load_query_constructor_chain    | load_query_constructor_runnable   | Use LCEL under the hood                        |
| VectorDBQA                      | RetrievalQA                       | More general to all retrievers                 |
| Sequential Chain                | LCEL                              | Obviated by LCEL                               |
| SimpleSequentialChain           | LCEL                              | Obviated by LCEL                               |
| TransformChain                  | LCEL/RunnableLambda               | Obviated by LCEL                               |
| create_tagging_chain            | create_structured_output_runnable | Use LCEL under the hood                        |
| ChatAgent                       | create_react_agent                | Use LCEL builder over a class                  |
| ConversationalAgent             | create_react_agent                | Use LCEL builder over a class                  |
| ConversationalChatAgent         | create_json_chat_agent            | Use LCEL builder over a class                  |
| initialize_agent                | Individual create agent methods   | Individual create agent methods are more clear |
| ZeroShotAgent                   | create_react_agent                | Use LCEL builder over a class                  |
| OpenAIFunctionsAgent            | create_openai_functions_agent     | Use LCEL builder over a class                  |
| OpenAIMultiFunctionsAgent       | create_openai_tools_agent         | Use LCEL builder over a class                  |
| SelfAskWithSearchAgent          | create_self_ask_with_search       | Use LCEL builder over a class                  |
| StructuredChatAgent             | create_structured_chat_agent      | Use LCEL builder over a class                  |
| XMLAgent                        | create_xml_agent                  | Use LCEL builder over a class                  |


# langchain-core

## 0.1.x

#### Deprecated

- `BaseChatModel` methods `__call__`, `call_as_llm`, `predict`, `predict_messages`. Will be removed in 0.2.0. Use `BaseChatModel.invoke` instead.
- `BaseChatModel` methods `apredict`, `apredict_messages`. Will be removed in 0.2.0. Use `BaseChatModel.ainvoke` instead.
- `BaseLLM` methods `__call__, `predict`, `predict_messages`. Will be removed in 0.2.0. Use `BaseLLM.invoke` instead.
- `BaseLLM` methods `apredict`, `apredict_messages`. Will be removed in 0.2.0. Use `BaseLLM.ainvoke` instead.


---
sidebar_position: 2
sidebar_label: Release policy
---

# LangChain release policy

The LangChain ecosystem is composed of different component packages (e.g., `langchain-core`, `langchain`, `langchain-community`, `langgraph`, `langserve`, partner packages etc.)

## Versioning

### `langchain`, `langchain-core`, and integration packages

`langchain`, `langchain-core`, `langchain-text-splitters`, and integration packages (`langchain-openai`, `langchain-anthropic`, etc.) follow [semantic versioning](https://semver.org/) in the format of 0.**Y**.**Z**. The packages are under rapid development, and so are currently versioning the packages with a major version of 0.

Minor version increases will occur for:

- Breaking changes for any public interfaces *not* marked as `beta`.

Patch version increases will occur for:

- Bug fixes,
- New features,
- Any changes to private interfaces,
- Any changes to `beta` features.

When upgrading between minor versions, users should review the list of breaking changes and deprecations.

From time to time, we will version packages as **release candidates**. These are versions that are intended to be released as stable versions, but we want to get feedback from the community before doing so. Release candidates will be versioned as 0.**Y**.**Z**rc**N**. For example, 0.2.0rc1. If no issues are found, the release candidate will be released as a stable version with the same version number. If issues are found, we will release a new release candidate with an incremented `N` value (e.g., 0.2.0rc2).

### `langchain-community`

`langchain-community` is currently on version `0.2.x`.

Minor version increases will occur for:

- Updates to the major/minor versions of required `langchain-x` dependencies. E.g., when updating the required version of `langchain-core` from `^0.2.x` to `0.3.0`.

Patch version increases will occur for:

- Bug fixes,
- New features,
- Any changes to private interfaces,
- Any changes to `beta` features,
- Breaking changes to integrations to reflect breaking changes in the third-party service.

Whenever possible we will avoid making breaking changes in patch versions.
However, if an external API makes a breaking change then breaking changes to the corresponding `langchain-community` integration can occur in a patch version.

### `langchain-experimental`

`langchain-experimental` is currently on version `0.0.x`. All changes will be accompanied with patch version increases.

## Release cadence

We expect to space out **minor** releases (e.g., from 0.2.x to 0.3.0) of `langchain` and `langchain-core` by at least 2-3 months, as such releases may contain breaking changes.

Patch versions are released frequently, up to a few times per week, as they contain bug fixes and new features.

## API stability

The development of LLM applications is a rapidly evolving field, and we are constantly learning from our users and the community. As such, we expect that the APIs in `langchain` and `langchain-core` will continue to evolve to better serve the needs of our users.

Even though both `langchain` and `langchain-core` are currently in a pre-1.0 state, we are committed to maintaining API stability in these packages.

- Breaking changes to the public API will result in a minor version bump (the second digit)
- Any bug fixes or new features will result in a patch version bump (the third digit)

We will generally try to avoid making unnecessary changes, and will provide a deprecation policy for features that are being removed.

### Stability of other packages

The stability of other packages in the LangChain ecosystem may vary:

- `langchain-community` is a community maintained package that contains 3rd party integrations. While we do our best to review and test changes in `langchain-community`, `langchain-community` is expected to experience more breaking changes than `langchain` and `langchain-core` as it contains many community contributions.
- Partner packages may follow different stability and versioning policies, and users should refer to the documentation of those packages for more information; however, in general these packages are expected to be stable.

### What is a "API stability"?

API stability means:

- All the public APIs (everything in this documentation) will not be moved or renamed without providing backwards-compatible aliases.
- If new features are added to these APIs – which is quite possible – they will not break or change the meaning of existing methods. In other words, "stable" does not (necessarily) mean "complete."
- If, for some reason, an API declared stable must be removed or replaced, it will be declared deprecated but will remain in the API for at least two minor releases. Warnings will be issued when the deprecated method is called.

### **APIs marked as internal**

Certain APIs are explicitly marked as “internal” in a couple of ways:

- Some documentation refers to internals and mentions them as such. If the documentation says that something is internal, it may change.
- Functions, methods, and other objects prefixed by a leading underscore (**`_`**). This is the standard Python convention of indicating that something is private; if any method starts with a single **`_`**, it’s an internal API.
    - **Exception:** Certain methods are prefixed with `_` , but do not contain an implementation. These methods are *meant* to be overridden by sub-classes that provide the implementation. Such methods are generally part of the **Public API** of LangChain.

## Deprecation policy

We will generally avoid deprecating features until a better alternative is available.

When a feature is deprecated, it will continue to work in the current and next minor version of `langchain` and `langchain-core`. After that, the feature will be removed.

Since we're expecting to space out minor releases by at least 2-3 months, this means that a feature can be removed within 2-6 months of being deprecated.

In some situations, we may allow the feature to remain in the code base for longer periods of time, if it's not causing issues in the packages, to reduce the burden on users.


# LangChain v0.3

*Last updated: 09.16.24*

## What's changed

* All packages have been upgraded from Pydantic 1 to Pydantic 2 internally. Use of Pydantic 2 in user code is fully supported with all packages without the need for bridges like `langchain_core.pydantic_v1` or `pydantic.v1`.
* Pydantic 1 will no longer be supported as it reached its end-of-life in June 2024.
* Python 3.8 will no longer be supported as its end-of-life is October 2024.

**These are the only breaking changes.**

## What’s new

The following features have been added during the development of 0.2.x:

- Moved more integrations from `langchain-community` to their own `langchain-x` packages. This is a non-breaking change, as the legacy implementations are left in `langchain-community` and marked as deprecated. This allows us to better manage the dependencies of, test, and version these integrations. You can see all the latest integration packages in the [API reference](https://python.langchain.com/v0.2/api_reference/reference.html#integrations).
- Simplified tool definition and usage. Read more [here](https://blog.langchain.dev/improving-core-tool-interfaces-and-docs-in-langchain/).
- Added utilities for interacting with chat models: [universal model constructor](https://python.langchain.com/v0.2/docs/how_to/chat_models_universal_init/), [rate limiter](https://python.langchain.com/v0.2/docs/how_to/chat_model_rate_limiting/), [message utilities](https://python.langchain.com/v0.2/docs/how_to/#messages),
- Added the ability to [dispatch custom events](https://python.langchain.com/v0.2/docs/how_to/callbacks_custom_events/).
- Revamped integration docs and API reference. Read more [here](https://blog.langchain.dev/langchain-integration-docs-revamped/).
- Marked as deprecated a number of legacy chains and added migration guides for all of them. These are slated for removal in `langchain` 1.0.0. See the deprecated chains and associated [migration guides here](https://python.langchain.com/v0.2/docs/versions/migrating_chains/).

## How to update your code

If you're using `langchain` / `langchain-community` / `langchain-core` 0.0 or 0.1, we recommend that you first [upgrade to 0.2](https://python.langchain.com/v0.2/docs/versions/v0_2/).

If you're using `langgraph`, upgrade to `langgraph>=0.2.20,<0.3`. This will work with either 0.2 or 0.3 versions of all the base packages.

Here is a complete list of all packages that have been released and what we recommend upgrading your version constraints to.
Any package that now requires `langchain-core` 0.3 had a minor version bump.
Any package that is now compatible with both `langchain-core` 0.2 and 0.3 had a patch version bump.

You can use the `langchain-cli` to update deprecated imports automatically.
The CLI will handle updating deprecated imports that were introduced in LangChain 0.0.x and LangChain 0.1, as
well as updating the `langchain_core.pydantic_v1` and `langchain.pydantic_v1` imports.


### Base packages

| Package                  | Latest | Recommended constraint |
|--------------------------|--------|------------------------|
| langchain                | 0.3.0  | >=0.3,<0.4             |
| langchain-community      | 0.3.0  | >=0.3,<0.4             |
| langchain-text-splitters | 0.3.0  | >=0.3,<0.4             |
| langchain-core           | 0.3.0  | >=0.3,<0.4             |
| langchain-experimental   | 0.3.0  | >=0.3,<0.4             |

### Downstream packages

| Package   | Latest | Recommended constraint |
|-----------|--------|------------------------|
| langgraph | 0.2.20 | >=0.2.20,<0.3          |
| langserve | 0.3.0  | >=0.3,<0.4             |

### Integration packages

| Package                                | Latest  | Recommended constraint     |
| -------------------------------------- | ------- | -------------------------- |
| langchain-ai21                         | 0.2.0   | >=0.2,<0.3                 |
| langchain-aws                          | 0.2.0   | >=0.2,<0.3                 |
| langchain-anthropic                    | 0.2.0   | >=0.2,<0.3                 |
| langchain-astradb                      | 0.4.1   | >=0.4.1,<0.5               |
| langchain-azure-dynamic-sessions       | 0.2.0   | >=0.2,<0.3                 |
| langchain-box                          | 0.2.0   | >=0.2,<0.3                 |
| langchain-chroma                       | 0.1.4   | >=0.1.4,<0.2               |
| langchain-cohere                       | 0.3.0   | >=0.3,<0.4                 |
| langchain-elasticsearch                | 0.3.0   | >=0.3,<0.4                 |
| langchain-exa                          | 0.2.0   | >=0.2,<0.3                 |
| langchain-fireworks                    | 0.2.0   | >=0.2,<0.3                 |
| langchain-groq                         | 0.2.0   | >=0.2,<0.3                 |
| langchain-google-community             | 2.0.0   | >=2,<3                     |
| langchain-google-genai                 | 2.0.0   | >=2,<3                     |
| langchain-google-vertexai              | 2.0.0   | >=2,<3                     |
| langchain-huggingface                  | 0.1.0   | >=0.1,<0.2                 |
| langchain-ibm                          | 0.2.0   | >=0.2,<0.3                 |
| langchain-milvus                       | 0.1.6   | >=0.1.6,<0.2               |
| langchain-mistralai                    | 0.2.0   | >=0.2,<0.3                 |
| langchain-mongodb                      | 0.2.0   | >=0.2,<0.3                 |
| langchain-nomic                        | 0.1.3   | >=0.1.3,<0.2               |
| langchain-ollama                       | 0.2.0   | >=0.2,<0.3                 |
| langchain-openai                       | 0.2.0   | >=0.2,<0.3                 |
| langchain-pinecone                     | 0.2.0   | >=0.2,<0.3                 |
| langchain-postgres                     | 0.0.13  | >=0.0.13,<0.1              |
| langchain-prompty                      | 0.1.0   | >=0.1,<0.2                 |
| langchain-qdrant                       | 0.1.4   | >=0.1.4,<0.2               |
| langchain-redis                        | 0.1.0   | >=0.1,<0.2                 |
| langchain-sema4                        | 0.2.0   | >=0.2,<0.3                 |
| langchain-together                     | 0.2.0   | >=0.2,<0.3                 |
| langchain-unstructured                 | 0.1.4   | >=0.1.4,<0.2               |
| langchain-upstage                      | 0.3.0   | >=0.3,<0.4                 |
| langchain-voyageai                     | 0.2.0   | >=0.2,<0.3                 |
| langchain-weaviate                     | 0.0.3   | >=0.0.3,<0.1               |

Once you've updated to recent versions of the packages, you may need to address the following issues stemming from the internal switch from Pydantic v1 to Pydantic v2:

- If your code depends on Pydantic aside from LangChain, you will need to upgrade your pydantic version constraints to be `pydantic>=2,<3`.  See [Pydantic’s migration guide](https://docs.pydantic.dev/latest/migration/) for help migrating your non-LangChain code to Pydantic v2 if you use pydantic v1.
- There are a number of side effects to LangChain components caused by the internal switch from Pydantic v1 to v2. We have listed some of the common cases below together with the recommended solutions.

## Common issues when transitioning to Pydantic 2

### 1. Do not use the `langchain_core.pydantic_v1` namespace

Replace any usage of `langchain_core.pydantic_v1` or `langchain.pydantic_v1` with
direct imports from `pydantic`.

For example,

```python
from langchain_core.pydantic_v1 import BaseModel
```

to:

```python
from pydantic import BaseModel
```

This may require you to make additional updates to your Pydantic code given that there are a number of breaking changes in Pydantic 2. See the [Pydantic Migration](https://docs.pydantic.dev/latest/migration/) for how to upgrade your code from Pydantic 1 to 2.

### 2. Passing Pydantic objects to LangChain APIs

Users using the following APIs:

* `BaseChatModel.bind_tools`
* `BaseChatModel.with_structured_output`
* `Tool.from_function`
* `StructuredTool.from_function`

should ensure that they are passing Pydantic 2 objects to these APIs rather than
Pydantic 1 objects (created via the `pydantic.v1` namespace of pydantic 2).

:::caution
While `v1` objets may be accepted by some of these APIs, users are advised to
use Pydantic 2 objects to avoid future issues.
:::

### 3. Sub-classing LangChain models

Any sub-classing from existing LangChain models (e.g., `BaseTool`, `BaseChatModel`, `LLM`)
should upgrade to use Pydantic 2 features.

For example, any user code that's relying on Pydantic 1 features (e.g., `validator`) should
be updated to the Pydantic 2 equivalent (e.g., `field_validator`), and any references to
`pydantic.v1`, `langchain_core.pydantic_v1`, `langchain.pydantic_v1` should be replaced
with imports from `pydantic`.

```python
from pydantic.v1 import validator, Field # if pydantic 2 is installed
# from pydantic import validator, Field # if pydantic 1 is installed
# from langchain_core.pydantic_v1 import validator, Field
# from langchain.pydantic_v1 import validator, Field

class CustomTool(BaseTool): # BaseTool is v1 code
    x: int = Field(default=1)

    def _run(*args, **kwargs):
        return "hello"

    @validator('x') # v1 code
    @classmethod
    def validate_x(cls, x: int) -> int:
        return 1
```

Should change to:

```python
from pydantic import Field, field_validator # pydantic v2
from langchain_core.pydantic_v1 import BaseTool

class CustomTool(BaseTool): # BaseTool is v1 code
    x: int = Field(default=1)

    def _run(*args, **kwargs):
        return "hello"


CustomTool(
    name='custom_tool',
    description="hello",
    x=1,
)
```

### 4. model_rebuild()

When sub-classing from LangChain models, users may need to add relevant imports
to the file and rebuild the model.

You can read more about `model_rebuild` [here](https://docs.pydantic.dev/latest/concepts/models/#rebuilding-model-schema).

```python
from langchain_core.output_parsers import BaseOutputParser

class FooParser(BaseOutputParser):
    ...
```

New code:

```python
from typing import Optional as Optional

from langchain_core.output_parsers import BaseOutputParser

class FooParser(BaseOutputParser):
    ...

FooParser.model_rebuild()
```

## Migrate using langchain-cli

The `langchain-cli` can help update deprecated LangChain imports in your code automatically.

Please note that the `langchain-cli` only handles deprecated LangChain imports and cannot
help to upgrade your code from pydantic 1 to pydantic 2.

For help with the Pydantic 1 to 2 migration itself please refer to the [Pydantic Migration Guidelines](https://docs.pydantic.dev/latest/migration/).

As of 0.0.31, the `langchain-cli` relies on [gritql](https://about.grit.io/) for applying code mods.

### Installation

```bash
pip install -U langchain-cli
langchain-cli --version # <-- Make sure the version is at least 0.0.31
```

### Usage

Given that the migration script is not perfect, you should make sure you have a backup of your code first (e.g., using version control like `git`).

The `langchain-cli` will handle the `langchain_core.pydantic_v1` deprecation introduced in LangChain 0.3 as well
as older deprecations (e.g.,`from langchain.chat_models import ChatOpenAI` which should be `from langchain_openai import ChatOpenAI`),

You will need to run the migration script **twice** as it only applies one import replacement per run.

For example, say that your code is still using the old import `from langchain.chat_models import ChatOpenAI`:

After the first run, you’ll get: `from langchain_community.chat_models import ChatOpenAI`
After the second run, you’ll get: `from langchain_openai import ChatOpenAI`

```bash
# Run a first time
# Will replace from langchain.chat_models import ChatOpenAI
langchain-cli migrate --help [path to code] # Help
langchain-cli migrate [path to code] # Apply

# Run a second time to apply more import replacements
langchain-cli migrate --diff [path to code] # Preview
langchain-cli migrate [path to code] # Apply
```

### Other options

```bash
# See help menu
langchain-cli migrate --help
# Preview Changes without applying
langchain-cli migrate --diff [path to code]
# Approve changes interactively
langchain-cli migrate --interactive [path to code]
```


---
sidebar_position: 0
---

# Overview

## What’s new in LangChain?

The following features have been added during the development of 0.1.x:

- Better streaming support via the [Event Streaming API](https://python.langchain.com/docs/expression_language/streaming/#using-stream-events).
- [Standardized tool calling support](https://blog.langchain.dev/tool-calling-with-langchain/)
- A standardized interface for [structuring output](https://github.com/langchain-ai/langchain/discussions/18154)
- [@chain decorator](https://python.langchain.com/docs/expression_language/how_to/decorator/) to more easily create **RunnableLambdas**
- https://python.langchain.com/docs/expression_language/how_to/inspect/
- In Python, better async support for many core abstractions (thank you [@cbornet](https://github.com/cbornet)!!)
- Include response metadata in `AIMessage` to make it easy to access raw output from the underlying models
- Tooling to visualize [your runnables](https://python.langchain.com/docs/expression_language/how_to/inspect/) or [your langgraph app](https://github.com/langchain-ai/langgraph/blob/main/examples/visualization.ipynb)
- Interoperability of chat message histories across most providers
- [Over 20+ partner packages in python](https://python.langchain.com/docs/integrations/providers/) for popular integrations

## What’s coming to LangChain?

- We’ve been working hard on [langgraph](https://langchain-ai.github.io/langgraph/). We will be building more capabilities on top of it and focusing on making it the go-to framework for agent architectures.
- Vectorstores V2! We’ll be revisiting our vectorstores abstractions to help improve usability and reliability.
- Better documentation and versioned docs!
- We’re planning a breaking release (0.3.0) sometime between July-September to [upgrade to full support of Pydantic 2](https://github.com/langchain-ai/langchain/discussions/19339), and will drop support for Pydantic 1 (including objects originating from the `v1` namespace of Pydantic 2).

## What changed?

Due to the rapidly evolving field, LangChain has also evolved rapidly.

This document serves to outline at a high level what has changed and why.

### TLDR

**As of 0.2.0:**

- This release completes the work that we started with release 0.1.0 by removing the dependency of `langchain` on `langchain-community`.
- `langchain` package no longer requires `langchain-community` . Instead `langchain-community` will now depend on `langchain-core` and `langchain` .
- User code that still relies on deprecated imports from `langchain` will continue to work as long `langchain_community` is installed. These imports will start raising errors in release 0.4.x.

**As of 0.1.0:**

- `langchain` was split into the following component packages: `langchain-core`, `langchain`, `langchain-community`, `langchain-[partner]` to improve the usability of langchain code in production settings. You can read more about it on our [blog](https://blog.langchain.dev/langchain-v0-1-0/).

### Ecosystem organization

By the release of 0.1.0, LangChain had grown to a large ecosystem with many integrations and a large community.

To improve the usability of LangChain in production, we split the single `langchain` package into multiple packages. This allowed us to create a good foundation architecture for the LangChain ecosystem and improve the usability of `langchain` in production.

Here is the high level break down of the Eco-system:

- **langchain-core**:  contains core abstractions involving LangChain Runnables, tooling for observability, and base implementations of important abstractions (e.g., Chat Models).
- **langchain:** contains generic code that is built using interfaces defined in `langchain-core`. This package is for code that generalizes well across different implementations of specific interfaces. For example, `create_tool_calling_agent` works across chat models that support [tool calling capabilities](https://blog.langchain.dev/tool-calling-with-langchain/).
- **langchain-community**: community maintained 3rd party integrations. Contains integrations based on interfaces defined in **langchain-core**. Maintained by the LangChain community.
- **Partner Packages (e.g., langchain-[partner])**: Partner packages are packages dedicated to especially popular integrations (e.g., `langchain-openai`, `langchain-anthropic` etc.). The dedicated packages generally benefit from better reliability and support.
- `langgraph`: Build robust and stateful multi-actor applications with LLMs by modeling steps as edges and nodes in a graph.
- `langserve`: Deploy LangChain chains as REST APIs.

In the 0.1.0 release, `langchain-community` was retained as required a dependency of `langchain`.

This allowed imports of vectorstores, chat models, and other integrations to continue working through `langchain`
rather than forcing users to update all of their imports to `langchain-community`.

For the 0.2.0 release, we’re removing the dependency of `langchain` on `langchain-community`. This is something we’ve been planning to do since the 0.1 release because we believe this is the right package architecture.

Old imports will continue to work as long as `langchain-community` is installed. These imports will be removed in the 0.4.0 release.

To understand why we think breaking the dependency of `langchain` on `langchain-community` is best we should understand what each package is meant to do.

`langchain` is meant to contain high-level chains and agent architectures. The logic in these should be specified at the level of abstractions like `ChatModel` and `Retriever`, and should not be specific to any one integration. This has two main benefits:

1. `langchain` is fairly lightweight. Here is the full list of required dependencies (after the split)

    ```toml
    python = ">=3.8.1,<4.0"
    langchain-core = "^0.2.0"
    langchain-text-splitters = ">=0.0.1,<0.1"
    langsmith = "^0.1.17"
    pydantic = ">=1,<3"
    SQLAlchemy = ">=1.4,<3"
    requests = "^2"
    PyYAML = ">=5.3"
    numpy = "^1"
    aiohttp = "^3.8.3"
    tenacity = "^8.1.0"
    jsonpatch = "^1.33"
    ```

2. `langchain` chains/agents are largely integration-agnostic, which makes it easy to experiment with different integrations and future-proofs your code should there be issues with one specific integration.

There is also a third less tangible benefit which is that being integration-agnostic forces us to find only those very generic abstractions and architectures which generalize well across integrations. Given how general the abilities of the foundational tech are, and how quickly the space is moving, having generic architectures is a good way of future-proofing your applications.

`langchain-community` is intended to have all integration-specific components that are not yet being maintained in separate `langchain-{partner}` packages. Today this is still the majority of integrations and a lot of code. This code is primarily contributed by the community, while `langchain` is largely written by core maintainers. All of these integrations use optional dependencies and conditional imports, which prevents dependency bloat and conflicts but means compatible dependency versions are not made explicit. Given the volume of integrations in `langchain-community` and the speed at which integrations change, it’s very hard to follow semver versioning, and we currently don’t.

All of which is to say that there’s no large benefits to `langchain` depending on `langchain-community` and some obvious downsides: the functionality in `langchain` should be integration agnostic anyways, `langchain-community` can’t be properly versioned, and depending on `langchain-community` increases the [vulnerability surface](https://github.com/langchain-ai/langchain/discussions/19083) of `langchain`.

For more context about the reason for the organization please see our blog: https://blog.langchain.dev/langchain-v0-1-0/


---
sidebar_position: 1
---

# Migration

LangChain v0.2 was released in May 2024. This release includes a number of [breaking changes and deprecations](/docs/versions/v0_2/deprecations). This document contains a guide on upgrading to 0.2.x.

:::note Reference

- [Breaking Changes & Deprecations](/docs/versions/v0_2/deprecations)
- [Migrating legacy chains to LCEL](/docs/versions/migrating_chains)
- [Migrating to Astream Events v2](/docs/versions/v0_2/migrating_astream_events)

:::

# Migration

This documentation will help you upgrade your code to LangChain `0.2.x.`. To prepare for migration, we first recommend you take the following steps:

1. Install the 0.2.x versions of langchain-core, langchain and upgrade to recent versions of other packages that you may be using. (e.g. langgraph, langchain-community, langchain-openai, etc.)
2. Verify that your code runs properly with the new packages (e.g., unit tests pass).
3. Install a recent version of `langchain-cli` , and use the tool to replace old imports used by your code with the new imports. (See instructions below.)
4. Manually resolve any remaining deprecation warnings.
5. Re-run unit tests.
6. If you are using `astream_events`, please review how to [migrate to astream events v2](/docs/versions/v0_2/migrating_astream_events).

## Upgrade to new imports

We created a tool to help migrate your code. This tool is still in **beta** and may not cover all cases, but
we hope that it will help you migrate your code more quickly.

The migration script has the following limitations:

1. It’s limited to helping users move from old imports to new imports. It does not help address other deprecations.
2. It can’t handle imports that involve `as` .
3. New imports are always placed in global scope, even if the old import that was replaced was located inside some local scope (e..g, function body).
4. It will likely miss some deprecated imports.

Here is an example of the import changes that the migration script can help apply automatically:

| From Package        | To Package               | Deprecated Import                                                  | New Import                                                          |
|---------------------|--------------------------|--------------------------------------------------------------------|---------------------------------------------------------------------|
| langchain           | langchain-community      | from langchain.vectorstores import InMemoryVectorStore             | from langchain_community.vectorstores import InMemoryVectorStore    |
| langchain-community | langchain_openai         | from langchain_community.chat_models import ChatOpenAI             | from langchain_openai import ChatOpenAI                             |
| langchain-community | langchain-core           | from langchain_community.document_loaders import Blob              | from langchain_core.document_loaders import Blob                    |
| langchain           | langchain-core           | from langchain.schema.document import Document                     | from langchain_core.documents import Document                       |
| langchain           | langchain-text-splitters | from langchain.text_splitter import RecursiveCharacterTextSplitter | from langchain_text_splitters import RecursiveCharacterTextSplitter |

## Installation

```bash
pip install langchain-cli
langchain-cli --version # <-- Make sure the version is at least 0.0.22
```

## Usage

Given that the migration script is not perfect, you should make sure you have a backup of your code first (e.g., using version control like `git`).

You will need to run the migration script **twice** as it only applies one import replacement per run.

For example, say your code still uses `from langchain.chat_models import ChatOpenAI`:

After the first run, you’ll get: `from langchain_community.chat_models import ChatOpenAI`
After the second run, you’ll get: `from langchain_openai import ChatOpenAI`

```bash
# Run a first time
# Will replace from langchain.chat_models import ChatOpenAI
langchain-cli migrate --diff [path to code] # Preview
langchain-cli migrate [path to code] # Apply

# Run a second time to apply more import replacements
langchain-cli migrate --diff [path to code] # Preview
langchain-cli migrate [path to code] # Apply
```

### Other options

```bash
# See help menu
langchain-cli migrate --help
# Preview Changes without applying
langchain-cli migrate --diff [path to code]
# Run on code including ipython notebooks
# Apply all import updates except for updates from langchain to langchain-core
langchain-cli migrate --disable langchain_to_core --include-ipynb [path to code]
```


---
sidebar_position: 3
sidebar_label: Changes
keywords: [retrievalqa, llmchain, conversationalretrievalchain]
---

# Deprecations and Breaking Changes

This code contains a list of deprecations and removals in the `langchain` and `langchain-core` packages.

New features and improvements are not listed here. See the [overview](/docs/versions/v0_2/overview/) for a summary of what's new in this release.

## Breaking changes

As of release 0.2.0, `langchain` is required to be integration-agnostic. This means that code in `langchain`  should not by default instantiate any specific chat models, llms, embedding models, vectorstores etc; instead, the user will be required to specify those explicitly.

The following functions and classes require an explicit LLM to be passed as an argument:

- `langchain.agents.agent_toolkits.vectorstore.toolkit.VectorStoreToolkit`
- `langchain.agents.agent_toolkits.vectorstore.toolkit.VectorStoreRouterToolkit`
- `langchain.chains.openai_functions.get_openapi_chain`
- `langchain.chains.router.MultiRetrievalQAChain.from_retrievers`
- `langchain.indexes.VectorStoreIndexWrapper.query`
- `langchain.indexes.VectorStoreIndexWrapper.query_with_sources`
- `langchain.indexes.VectorStoreIndexWrapper.aquery_with_sources`
- `langchain.chains.flare.FlareChain`

The following classes now require passing an explicit Embedding model as an argument:

- `langchain.indexes.VectostoreIndexCreator`

The following code has been removed:

- `langchain.natbot.NatBotChain.from_default` removed in favor of the `from_llm` class method.

Behavior was changed for the following code:

### @tool decorator

`@tool` decorator now assigns the function doc-string as the tool description. Previously, the `@tool` decorator
using to prepend the function signature.

Before 0.2.0:

```python
@tool
def my_tool(x: str) -> str:
    """Some description."""
    return "something"

print(my_tool.description)
```

Would result in: `my_tool: (x: str) -> str - Some description.`

As of 0.2.0:

It will result in: `Some description.`

## Code that moved to another package

Code that was moved from `langchain` into another package (e.g, `langchain-community`)

If you try to import it from `langchain`, the import will keep on working, but will raise a deprecation warning. The warning will provide a replacement import statement.

 ```shell
 python -c "from langchain.document_loaders.markdown import UnstructuredMarkdownLoader"
```

 ```shell
 LangChainDeprecationWarning: Importing UnstructuredMarkdownLoader from langchain.document_loaders is deprecated. Please replace deprecated imports:

 >> from langchain.document_loaders import UnstructuredMarkdownLoader

 with new imports of:

 >> from langchain_community.document_loaders import UnstructuredMarkdownLoader
```

We will continue supporting the imports in `langchain` until release 0.4 as long as the relevant package where the code lives is installed. (e.g., as long as `langchain_community` is installed.)

However, we advise for users to not rely on these imports and instead migrate to the new imports. To help with this process, we’re releasing a migration script via the LangChain CLI. See further instructions in migration guide.

## Code targeted for removal

Code that has better alternatives available and will eventually be removed, so there’s only a single way to do things. (e.g., `predict_messages` method in ChatModels has been deprecated in favor of `invoke`).

### astream events V1

If you are using `astream_events`, please review how to [migrate to astream events v2](/docs/versions/v0_2/migrating_astream_events).

### langchain_core

#### try_load_from_hub

In module: `utils.loading`
Deprecated: 0.1.30
Removal: 0.3.0

Alternative: Using the hwchase17/langchain-hub repo for prompts is deprecated. Please use https://smith.langchain.com/hub instead.

#### BaseLanguageModel.predict

In module: `language_models.base`
Deprecated: 0.1.7
Removal: 0.3.0

Alternative: invoke

#### BaseLanguageModel.predict_messages

In module: `language_models.base`
Deprecated: 0.1.7
Removal: 0.3.0

Alternative: invoke

#### BaseLanguageModel.apredict

In module: `language_models.base`
Deprecated: 0.1.7
Removal: 0.3.0

Alternative: ainvoke

#### BaseLanguageModel.apredict_messages

In module: `language_models.base`
Deprecated: 0.1.7
Removal: 0.3.0

Alternative: ainvoke

#### RunTypeEnum

In module: `tracers.schemas`
Deprecated: 0.1.0
Removal: 0.3.0

Alternative: Use string instead.

#### TracerSessionV1Base

In module: `tracers.schemas`
Deprecated: 0.1.0
Removal: 0.3.0

Alternative:

#### TracerSessionV1Create

In module: `tracers.schemas`
Deprecated: 0.1.0
Removal: 0.3.0

Alternative:

#### TracerSessionV1

In module: `tracers.schemas`
Deprecated: 0.1.0
Removal: 0.3.0

Alternative:

#### TracerSessionBase

In module: `tracers.schemas`
Deprecated: 0.1.0
Removal: 0.3.0

Alternative:

#### TracerSession

In module: `tracers.schemas`
Deprecated: 0.1.0
Removal: 0.3.0

Alternative:

#### BaseRun

In module: `tracers.schemas`
Deprecated: 0.1.0
Removal: 0.3.0

Alternative: Run

#### LLMRun

In module: `tracers.schemas`
Deprecated: 0.1.0
Removal: 0.3.0

Alternative: Run

#### ChainRun

In module: `tracers.schemas`
Deprecated: 0.1.0
Removal: 0.3.0

Alternative: Run

#### ToolRun

In module: `tracers.schemas`
Deprecated: 0.1.0
Removal: 0.3.0

Alternative: Run

#### BaseChatModel.__call__

In module: `language_models.chat_models`
Deprecated: 0.1.7
Removal: 0.3.0

Alternative: invoke

#### BaseChatModel.call_as_llm

In module: `language_models.chat_models`
Deprecated: 0.1.7
Removal: 0.3.0

Alternative: invoke

#### BaseChatModel.predict

In module: `language_models.chat_models`
Deprecated: 0.1.7
Removal: 0.3.0

Alternative: invoke

#### BaseChatModel.predict_messages

In module: `language_models.chat_models`
Deprecated: 0.1.7
Removal: 0.3.0

Alternative: invoke

#### BaseChatModel.apredict

In module: `language_models.chat_models`
Deprecated: 0.1.7
Removal: 0.3.0

Alternative: ainvoke

#### BaseChatModel.apredict_messages

In module: `language_models.chat_models`
Deprecated: 0.1.7
Removal: 0.3.0

Alternative: ainvoke

#### BaseLLM.__call__

In module: `language_models.llms`
Deprecated: 0.1.7
Removal: 0.3.0

Alternative: invoke

#### BaseLLM.predict

In module: `language_models.llms`
Deprecated: 0.1.7
Removal: 0.3.0

Alternative: invoke

#### BaseLLM.predict_messages

In module: `language_models.llms`
Deprecated: 0.1.7
Removal: 0.3.0

Alternative: invoke

#### BaseLLM.apredict

In module: `language_models.llms`
Deprecated: 0.1.7
Removal: 0.3.0

Alternative: ainvoke

#### BaseLLM.apredict_messages

In module: `language_models.llms`
Deprecated: 0.1.7
Removal: 0.3.0

Alternative: ainvoke

#### BaseRetriever.get_relevant_documents

In module: `retrievers`
Deprecated: 0.1.46
Removal: 0.3.0

Alternative: invoke

#### BaseRetriever.aget_relevant_documents

In module: `retrievers`
Deprecated: 0.1.46
Removal: 0.3.0

Alternative: ainvoke

#### ChatPromptTemplate.from_role_strings

In module: `prompts.chat`
Deprecated: 0.0.1
Removal:

Alternative: from_messages classmethod

#### ChatPromptTemplate.from_strings

In module: `prompts.chat`
Deprecated: 0.0.1
Removal:

Alternative: from_messages classmethod

#### BaseTool.__call__


{
 "cells": [
  {
   "cell_type": "markdown",
   "id": "ed78c53c-55ad-4ea2-9cc2-a39a1963c098",
   "metadata": {},
   "source": [
    "# Migrating from StuffDocumentsChain\n",
    "\n",
    "[StuffDocumentsChain](https://python.langchain.com/api_reference/langchain/chains/langchain.chains.combine_documents.stuff.StuffDocumentsChain.html) combines documents by concatenating them into a single context window. It is a straightforward and effective strategy for combining documents for question-answering, summarization, and other purposes.\n",
    "\n",
    "[create_stuff_documents_chain](https://python.langchain.com/api_reference/langchain/chains/langchain.chains.combine_documents.stuff.create_stuff_documents_chain.html) is the recommended alternative. It functions the same as `StuffDocumentsChain`, with better support for streaming and batch functionality. Because it is a simple combination of [LCEL primitives](/docs/concepts/#langchain-expression-language-lcel), it is also easier to extend and incorporate into other LangChain applications.\n",
    "\n",
    "Below we will go through both `StuffDocumentsChain` and `create_stuff_documents_chain` on a simple example for illustrative purposes.\n",
    "\n",
    "Let's first load a chat model:\n",
    "\n",
    "import ChatModelTabs from \"@theme/ChatModelTabs\";\n",
    "\n",
    "<ChatModelTabs customVarName=\"llm\" />"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 1,
   "id": "dac0bef2-9453-46f2-a893-f7569b6a0170",
   "metadata": {},
   "outputs": [],
   "source": [
    "# | output: false\n",
    "# | echo: false\n",
    "\n",
    "from langchain_openai import ChatOpenAI\n",
    "\n",
    "llm = ChatOpenAI(model=\"gpt-4o-mini\", temperature=0)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "d4022d03-7b5e-4c81-98ff-5b82a2a4eaae",
   "metadata": {},
   "source": [
    "## Example\n",
    "\n",
    "Let's go through an example where we analyze a set of documents. We first generate some simple documents for illustrative purposes:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 2,
   "id": "24fa0ba9-e245-47d1-bc2e-6286dd884117",
   "metadata": {},
   "outputs": [],
   "source": [
    "from langchain_core.documents import Document\n",
    "\n",
    "documents = [\n",
    "    Document(page_content=\"Apples are red\", metadata={\"title\": \"apple_book\"}),\n",
    "    Document(page_content=\"Blueberries are blue\", metadata={\"title\": \"blueberry_book\"}),\n",
    "    Document(page_content=\"Bananas are yelow\", metadata={\"title\": \"banana_book\"}),\n",
    "]"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "3a769128-205f-417d-a25d-519e7cb03be7",
   "metadata": {},
   "source": [
    "### Legacy\n",
    "\n",
    "<details open>\n",
    "\n",
    "Below we show an implementation with `StuffDocumentsChain`. We define the prompt template for a summarization task and instantiate a [LLMChain](https://python.langchain.com/api_reference/langchain/chains/langchain.chains.llm.LLMChain.html) object for this purpose. We define how documents are formatted into the prompt and ensure consistency among the keys in the various prompts."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 15,
   "id": "9734c0f3-64e7-4ae6-8578-df03b3dabb26",
   "metadata": {},
   "outputs": [],
   "source": [
    "from langchain.chains import LLMChain, StuffDocumentsChain\n",
    "from langchain_core.prompts import ChatPromptTemplate, PromptTemplate\n",
    "\n",
    "# This controls how each document will be formatted. Specifically,\n",
    "# it will be passed to `format_document` - see that function for more\n",
    "# details.\n",
    "document_prompt = PromptTemplate(\n",
    "    input_variables=[\"page_content\"], template=\"{page_content}\"\n",
    ")\n",
    "document_variable_name = \"context\"\n",
    "# The prompt here should take as an input variable the\n",
    "# `document_variable_name`\n",
    "prompt = ChatPromptTemplate.from_template(\"Summarize this content: {context}\")\n",
    "\n",
    "llm_chain = LLMChain(llm=llm, prompt=prompt)\n",
    "chain = StuffDocumentsChain(\n",
    "    llm_chain=llm_chain,\n",
    "    document_prompt=document_prompt,\n",
    "    document_variable_name=document_variable_name,\n",
    ")"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "0cb733bf-eb71-4fae-a8f4-d522924020cb",
   "metadata": {},
   "source": [
    "We can now invoke our chain:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 19,
   "id": "d7d1ce10-bbee-4cb0-879d-7de4f69191c4",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "'This content describes the colors of different fruits: apples are red, blueberries are blue, and bananas are yellow.'"
      ]
     },
     "execution_count": 19,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "result = chain.invoke(documents)\n",
    "result[\"output_text\"]"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 20,
   "id": "79b10d40-1521-433b-9026-6ec836ffeeb3",
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "{'input_documents': [Document(metadata={'title': 'apple_book'}, page_content='Apples are red'), Document(metadata={'title': 'blueberry_book'}, page_content='Blueberries are blue'), Document(metadata={'title': 'banana_book'}, page_content='Bananas are yelow')], 'output_text': 'This content describes the colors of different fruits: apples are red, blueberries are blue, and bananas are yellow.'}\n"
     ]
    }
   ],
   "source": [
    "for chunk in chain.stream(documents):\n",
    "    print(chunk)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "b4cb6a5b-37ea-48cc-a096-b948d3ff7e9f",
   "metadata": {},
   "source": [
    "</details>\n",
    "\n",
    "### LCEL\n",
    "\n",
    "<details open>\n",
    "\n",
    "Below we show an implementation using `create_stuff_documents_chain`:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 21,
   "id": "de38f27a-c648-44be-8c37-0a458c2920a9",
   "metadata": {},
   "outputs": [],
   "source": [
    "from langchain.chains.combine_documents import create_stuff_documents_chain\n",
    "from langchain_core.prompts import ChatPromptTemplate\n",
    "\n",
    "prompt = ChatPromptTemplate.from_template(\"Summarize this content: {context}\")\n",
    "chain = create_stuff_documents_chain(llm, prompt)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "9d0e6996-9bf8-4097-9c1a-1c539eac3ed1",
   "metadata": {},
   "source": [
    "Invoking the chain, we obtain a similar result as before:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 24,
   "id": "f2d2bdfb-3a6a-464b-b4c2-e4252b2e53a0",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "'This content describes the colors of different fruits: apples are red, blueberries are blue, and bananas are yellow.'"
      ]
     },
     "execution_count": 24,
     "metadata": {},