Examples

This directory contains a collection of examples that demonstrate how to use the TRL library for various applications. We provide both scripts for advanced use cases and notebooks for an easy start and interactive experimentation.

The notebooks are self-contained and can run on free Colab, while the scripts can run on single GPU, multi-GPU, or DeepSpeed setups.

Getting Started

Install TRL and additional dependencies as follows:

pip install --upgrade trl[quantization]

Check for additional optional dependencies here.

For scripts, you will also need an 🤗 Accelerate config (recommended for multi-gpu settings):

accelerate config # will prompt you to define the training configuration

This allows you to run scripts with accelerate launch in single or multi-GPU settings.

Notebooks

These notebooks are easier to run and are designed for quick experimentation with TRL. The list of notebooks can be found in the trl/examples/notebooks/ directory.

Notebook	Description	Open in Colab
sft_trl_lora_qlora.ipynb	Supervised Fine-Tuning (SFT) using QLoRA on free Colab
sft_qwen_vl.ipynb	Supervised Fine-Tuning (SFT) Qwen3-VL with QLoRA using TRL on free Colab
grpo_qwen3_vl.ipynb	GRPO Qwen3-VL with QLoRA using TRL on free Colab

Legacy / Older Notebooks

• best_of_n.ipynb: This notebook demonstrates how to use the “Best of N” sampling strategy using TRL when fine-tuning your model with PPO.
• gpt2-sentiment.ipynb: This notebook demonstrates how to reproduce the GPT2 imdb sentiment tuning example on a jupyter notebook.
• gpt2-sentiment-control.ipynb: This notebook demonstrates how to reproduce the GPT2 sentiment control example on a jupyter notebook.

Scripts

Scripts are maintained in the trl/scripts and examples/scripts directories. They show how to use different trainers such as SFTTrainer, PPOTrainer, DPOTrainer, GRPOTrainer, and more.

Distributed Training (for scripts)

You can run scripts on multiple GPUs with 🤗 Accelerate:

accelerate launch --config_file=examples/accelerate_configs/multi_gpu.yaml --num_processes {NUM_GPUS} path_to_script.py --all_arguments_of_the_script

For DeepSpeed ZeRO-{1,2,3}:

accelerate launch --config_file=examples/accelerate_configs/deepspeed_zero{1,2,3}.yaml --num_processes {NUM_GPUS} path_to_script.py --all_arguments_of_the_script

Adjust NUM_GPUS and --all_arguments_of_the_script as needed.