qwen2_5_omni_text_only_cleaned

This model was trained from scratch on an unknown dataset. It achieves the following results on the evaluation set:

• Loss: 0.7060
• Token Acc: 0.7820

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

• learning_rate: 0.0001
• train_batch_size: 2
• eval_batch_size: 1
• seed: 42
• distributed_type: multi-GPU
• num_devices: 8
• gradient_accumulation_steps: 4
• total_train_batch_size: 64
• total_eval_batch_size: 8
• optimizer: Use OptimizerNames.ADAMW_TORCH_FUSED with betas=(0.9,0.95) and epsilon=1e-08 and optimizer_args=No additional optimizer arguments
• lr_scheduler_type: cosine
• lr_scheduler_warmup_ratio: 0.05
• num_epochs: 3.0

Training results

Training Loss	Epoch	Step	Validation Loss	Token Acc
0.9239	0.2494	50	0.9490	0.7274
0.9050	0.4988	100	0.9236	0.7314
0.8713	0.7481	150	0.9009	0.7387
0.8508	0.9975	200	0.8731	0.7440
0.7631	1.2444	250	0.8440	0.7516
0.7303	1.4938	300	0.8100	0.7569
0.7075	1.7431	350	0.7822	0.7649
0.6881	1.9925	400	0.7527	0.7708
0.5829	2.2394	450	0.7351	0.7760
0.5541	2.4888	500	0.7163	0.7796
0.5534	2.7382	550	0.7081	0.7811
0.5633	2.9875	600	0.7060	0.7820
0.5633	3.0	603	0.7060	0.7820

Framework versions

• Transformers 4.57.0
• Pytorch 2.8.0+cu128
• Datasets 3.6.0
• Tokenizers 0.22.1