#
Using different models

`smolagents` provides a flexible framework that allows you to use various language models from different providers. This guide will show you how to use different model types with your agents.

##
Available model types

`smolagents` supports several model types out of the box:

1. InferenceClientModel: Uses Hugging Face’s Inference API to access models
2. TransformersModel: Runs models locally using the Transformers library
3. VLLMModel: Uses vLLM for fast inference with optimized serving
4. MLXModel: Optimized for Apple Silicon devices using MLX
5. LiteLLMModel: Provides access to hundreds of LLMs through LiteLLM
6. LiteLLMRouterModel: Distributes requests among multiple models
7. OpenAIServerModel: Provides access to any provider that implements an OpenAI-compatible API
8. AzureOpenAIServerModel: Uses Azure’s OpenAI service
9. AmazonBedrockServerModel: Connects to AWS Bedrock’s API

##
Using Google Gemini Models

As explained in the Google Gemini API documentation (https://ai.google.dev/gemini-api/docs/openai), Google provides an OpenAI-compatible API for Gemini models, allowing you to use the OpenAIServerModel with Gemini models by setting the appropriate base URL.

First, install the required dependencies:

    pip install smolagents[openai]

Then, get a Gemini API key and set it in your code:

    GEMINI_API_KEY = <YOUR-GEMINI-API-KEY>

Now, you can initialize the Gemini model using the `OpenAIServerModel` class and setting the `api_base` parameter to the Gemini API base URL:

    from smolagents import OpenAIServerModel

    model = OpenAIServerModel(
        model_id="gemini-2.0-flash",
        # Google Gemini OpenAI-compatible API base URL
        api_base="https://generativelanguage.googleapis.com/v1beta/openai/",
        api_key=GEMINI_API_KEY,
    )

##
Using OpenRouter Models

OpenRouter provides access to a wide variety of language models through a unified OpenAI-compatible API. You can use the OpenAIServerModel to connect to OpenRouter by setting the appropriate base URL.

First, install the required dependencies:

    pip install smolagents[openai]

Then, get an OpenRouter API key and set it in your code:

    OPENROUTER_API_KEY = <YOUR-OPENROUTER-API-KEY>

Now, you can initialize any model available on OpenRouter using the `OpenAIServerModel` class:

    from smolagents import OpenAIServerModel

    model = OpenAIServerModel(
        # You can use any model ID available on OpenRouter
        model_id="openai/gpt-4o",
        # OpenRouter API base URL
        api_base="https://openrouter.ai/api/v1",
        api_key=OPENROUTER_API_KEY,
    )

< > Update on GitHub