  Padding is often used in tuning LLM models by adding special tokens to
shorter training examples to match the length of the longest sequence in each
batch. While this ensures uniformity for batch processing, it introduces
inefficiencies by including irrelevant padding tokens in the computation and
wastes GPU resources. Hugging Face SFT trainer has always offered the option to
use packing to combine multiple training examples, allowing for maximal
utilization of GPU resources. However, up till now, it did not offer proper
masking of each packed training example. This capability has now been added to
Hugging Face Transformers 4.43. We analyse this new feature and show the
benefits across different variations of packing.
