ATSC-albert-base-v2-For-SemEval-2014-Task-4

This model is a fine-tuned version of albert/albert-base-v2 on an unknown dataset. It achieves the following results on the evaluation set:

• Loss: 1.0392
• Accurancy: 0.8490

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

• learning_rate: 2e-05
• train_batch_size: 16
• eval_batch_size: 16
• seed:
• optimizer: Use OptimizerNames.ADAMW_TORCH with betas=(0.9,0.999) and epsilon=1e-08 and optimizer_args=No additional optimizer arguments
• lr_scheduler_type: linear
• num_epochs: 54

Training results

Epoch	Training Loss	Validation Loss	Accuracy
1	0.8649	0.8954	0.6497
2	0.6734	0.5684	0.7775
3	0.5461	0.4641	0.8097
4	0.4142	0.4540	0.8275
5	0.3211	0.5946	0.8034
6	0.2437	0.4974	0.8329
7	0.1958	0.4916	0.8168
8	0.1601	0.6348	0.8275
9	0.1095	0.6533	0.8293
10	0.0885	0.7212	0.8204
11	0.0714	0.7217	0.8240
12	0.0597	0.7698	0.8266
13	0.0420	0.7946	0.8400
14	0.0566	0.8103	0.8418
15	0.0389	0.9175	0.8275
16	0.0357	1.1165	0.8266
17	0.0205	1.0199	0.8302
18	0.0207	0.9885	0.8391
19	0.0155	1.0372	0.8374
20	0.0250	1.1147	0.8365
21	0.0198	1.0150	0.8472
22	0.0210	1.1716	0.8356
23	0.0208	1.0894	0.8454
24	0.0222	1.1699	0.8382
25	0.0196	1.2378	0.8338
26	0.0166	0.9921	0.8490
27	0.0115	1.0392	0.8490
28	0.0126	1.3480	0.8311
29	0.0107	1.2037	0.8427
30	0.0128	1.0996	0.8427
31	0.0128	1.1347	0.8320
32	0.0088	1.2735	0.8356
33	0.0083	1.2403	0.8409
34	0.0094	1.2600	0.8418
35	0.0072	1.2430	0.8454
36	0.0106	1.2740	0.8391
37	0.0093	1.1836	0.8427
38	0.0074	1.2132	0.8454
39	0.0071	1.1983	0.8463
40	0.0062	1.2708	0.8409
41	0.0068	1.2093	0.8463
42	0.0055	1.2593	0.8445
43	0.0055	1.2497	0.8445
44	0.0055	1.2530	0.8463
45	0.0051	1.2546	0.8463
46	0.0052	1.2513	0.8463
47	0.0054	1.2679	0.8481
48	0.0053	1.2839	0.8463
49	0.0048	1.2922	0.8445
50	0.0050	1.3092	0.8409
51	0.0052	1.2977	0.8436
52	0.0051	1.3066	0.8427
53	0.0051	1.3056	0.8436
54	0.0001	1.3047	0.8436

Framework versions

• Transformers 4.48.2
• Pytorch 2.5.1+cu124
• Datasets 3.2.0
• Tokenizers 0.21.0