biasrm-paper-replication

This model is a fine-tuned version of FacebookAI/roberta-base on the None dataset. It achieves the following results on the evaluation set:

• Loss: 0.0059
• Mse: 0.0059

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

• learning_rate: 0.0001
• train_batch_size: 128
• eval_batch_size: 128
• seed: 42
• optimizer: Use OptimizerNames.ADAMW_TORCH with betas=(0.9,0.999) and epsilon=1e-08 and optimizer_args=No additional optimizer arguments
• lr_scheduler_type: linear
• num_epochs: 30

Training results

Training Loss	Epoch	Step	Validation Loss	Mse
0.0224	1.0	21	0.0140	0.0140
0.0145	2.0	42	0.0078	0.0078
0.0148	3.0	63	0.0119	0.0119
0.0106	4.0	84	0.0143	0.0143
0.0104	5.0	105	0.0074	0.0074
0.0068	6.0	126	0.0096	0.0096
0.0068	7.0	147	0.0096	0.0096
0.0061	8.0	168	0.0076	0.0076
0.0052	9.0	189	0.0077	0.0077
0.0056	10.0	210	0.0112	0.0112
0.0048	11.0	231	0.0056	0.0056
0.0042	12.0	252	0.0082	0.0082
0.0042	13.0	273	0.0091	0.0091
0.0043	14.0	294	0.0061	0.0061
0.0033	15.0	315	0.0064	0.0064
0.0033	16.0	336	0.0073	0.0073
0.0036	17.0	357	0.0085	0.0085
0.0032	18.0	378	0.0057	0.0057
0.0035	19.0	399	0.0054	0.0054
0.0029	20.0	420	0.0051	0.0051
0.0027	21.0	441	0.0057	0.0057
0.0027	22.0	462	0.0058	0.0058
0.0026	23.0	483	0.0053	0.0053
0.0025	24.0	504	0.0066	0.0066
0.0022	25.0	525	0.0069	0.0069
0.0024	26.0	546	0.0066	0.0066
0.0024	27.0	567	0.0057	0.0057
0.0023	28.0	588	0.0061	0.0061
0.0023	29.0	609	0.0061	0.0061
0.002	30.0	630	0.0059	0.0059

Framework versions

• Transformers 4.57.1
• Pytorch 2.8.0+cu126
• Datasets 4.0.0
• Tokenizers 0.22.1