ht-stmini-cls-v7_pretrain_tdso-m0drp0.0trp0.5-cssl-msm-bml

This model is a fine-tuned version of on an unknown dataset. It achieves the following results on the evaluation set:

• Loss: 1.3999
• Loss Spp: 0.1014
• Loss Gtsp: 0.0825
• Loss Cssl: 0.5385
• Loss Msm: 0.6774
• Macro F1 Gtsp: 0.4154

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

• learning_rate: 2e-05
• train_batch_size: 8
• eval_batch_size: 4
• seed: 42
• optimizer: Use adamw_torch with betas=(0.9,0.999) and epsilon=1e-08 and optimizer_args=No additional optimizer arguments
• lr_scheduler_type: linear
• lr_scheduler_warmup_steps: 15035
• training_steps: 300701

Training results

Training Loss	Epoch	Step	Validation Loss	Loss Spp	Loss Gtsp	Loss Cssl	Loss Msm	Macro F1 Gtsp
12.6896	0.0033	1000	13.2019	0.3070	0.2600	2.1012	10.5337	0.3590
8.242	0.0067	2000	8.8407	0.2710	0.2104	0.9851	7.3742	0.3430
3.3559	0.0100	3000	4.5082	0.2518	0.0873	0.7820	3.3872	0.4151
3.2096	0.0133	4000	3.3223	0.2174	0.0859	0.7016	2.3174	0.4162
3.4086	0.0166	5000	2.7384	0.1923	0.0839	0.6552	1.8070	0.4160
3.2841	0.0200	6000	2.2257	0.1901	0.0836	0.6118	1.3401	0.4163
1.2872	0.0233	7000	1.9375	0.1469	0.0801	0.6204	1.0900	0.4112
0.7888	0.0266	8000	1.4530	0.1286	0.0827	0.5779	0.6638	0.4165
0.6175	0.0299	9000	1.4505	0.1295	0.0829	0.5496	0.6884	0.4158
0.5566	0.0333	10000	1.4061	0.1104	0.0811	0.5587	0.6559	0.4113
0.5163	0.0366	11000	1.4074	0.1040	0.0840	0.5446	0.6748	0.4168
0.4852	0.0399	12000	1.3984	0.1022	0.0821	0.5377	0.6764	0.4153
0.0	0.0432	13000	nan	nan	nan	nan	nan	0.0054
0.0	0.0466	14000	nan	nan	nan	nan	nan	0.0052
0.0	0.0499	15000	nan	nan	nan	nan	nan	0.0050
0.0	0.0532	16000	nan	nan	nan	nan	nan	0.0053
0.0	0.0565	17000	nan	nan	nan	nan	nan	0.0054
0.0	0.0599	18000	nan	nan	nan	nan	nan	0.0054
0.0	0.0632	19000	nan	nan	nan	nan	nan	0.0053
0.0	0.0665	20000	nan	nan	nan	nan	nan	0.0053
0.0	0.0698	21000	nan	nan	nan	nan	nan	0.0053
0.0	0.0732	22000	nan	nan	nan	nan	nan	0.0053

Framework versions

• Transformers 4.46.0
• Pytorch 2.3.1+cu121
• Datasets 2.20.0
• Tokenizers 0.20.1