biasrm-contexts

This model is a fine-tuned version of FacebookAI/roberta-base on the None dataset. It achieves the following results on the evaluation set:

• Loss: 0.0080
• Mse: 0.0080

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

• learning_rate: 0.0001
• train_batch_size: 128
• eval_batch_size: 128
• seed: 42
• optimizer: Use OptimizerNames.ADAMW_TORCH with betas=(0.9,0.999) and epsilon=1e-08 and optimizer_args=No additional optimizer arguments
• lr_scheduler_type: linear
• num_epochs: 30

Training results

Training Loss	Epoch	Step	Validation Loss	Mse
0.0212	1.0	22	0.0099	0.0099
0.0167	2.0	44	0.0115	0.0115
0.0152	3.0	66	0.0067	0.0067
0.0123	4.0	88	0.0118	0.0118
0.0115	5.0	110	0.0057	0.0057
0.0097	6.0	132	0.0067	0.0067
0.008	7.0	154	0.0094	0.0094
0.0096	8.0	176	0.0063	0.0063
0.0071	9.0	198	0.0074	0.0074
0.0087	10.0	220	0.0073	0.0073
0.0072	11.0	242	0.0096	0.0096
0.0063	12.0	264	0.0061	0.0061
0.0055	13.0	286	0.0100	0.0100
0.0048	14.0	308	0.0064	0.0064
0.0052	15.0	330	0.0084	0.0084
0.004	16.0	352	0.0082	0.0082
0.0042	17.0	374	0.0114	0.0114
0.0037	18.0	396	0.0105	0.0105
0.0043	19.0	418	0.0074	0.0074
0.0036	20.0	440	0.0073	0.0073
0.0035	21.0	462	0.0087	0.0087
0.0033	22.0	484	0.0083	0.0083
0.0036	23.0	506	0.0078	0.0078
0.0026	24.0	528	0.0077	0.0077
0.0031	25.0	550	0.0079	0.0079
0.003	26.0	572	0.0084	0.0084
0.0026	27.0	594	0.0084	0.0084
0.0029	28.0	616	0.0073	0.0073
0.0026	29.0	638	0.0075	0.0075
0.003	30.0	660	0.0080	0.0080

Framework versions

• Transformers 4.57.1
• Pytorch 2.8.0+cu126
• Datasets 4.0.0
• Tokenizers 0.22.1