Hugging Face's logo

aiface
/
ModernBERT-large_massive_modernbert_large_crf_v1

Transformers
Safetensors
Generated from Trainer
Model card Files
xet
Community
aiface commited on
Commit
0f9c911
·
verified ·
1 Parent(s): 7bde8ee

Model save

Browse files
Files changed (5) hide show
  1. README.md +79 -0
  2. intent_report_test.txt +75 -0
  3. model.safetensors +1 -1
  4. model_predict_test.csv +0 -0
  5. slot_report_test.txt +59 -0
README.md ADDED
@@ -0,0 +1,79 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ ---
2
+ library_name: transformers
3
+ license: apache-2.0
4
+ base_model: answerdotai/ModernBERT-large
5
+ tags:
6
+ - generated_from_trainer
7
+ model-index:
8
+ - name: ModernBERT-large_massive_modernbert_large_crf_v1
9
+ results: []
10
+ ---
11
+
12
+ <!-- This model card has been generated automatically according to the information the Trainer had access to. You
13
+ should probably proofread and complete it, then remove this comment. -->
14
+
15
+ # ModernBERT-large_massive_modernbert_large_crf_v1
16
+
17
+ This model is a fine-tuned version of [answerdotai/ModernBERT-large](https://huggingface.co/answerdotai/ModernBERT-large) on an unknown dataset.
18
+ It achieves the following results on the evaluation set:
19
+ - Loss: 15.5718
20
+ - Slot P: 0.5398
21
+ - Slot R: 0.6408
22
+ - Slot F1: 0.5860
23
+ - Slot Exact Match: 0.6001
24
+ - Intent Acc: 0.7831
25
+
26
+ ## Model description
27
+
28
+ More information needed
29
+
30
+ ## Intended uses & limitations
31
+
32
+ More information needed
33
+
34
+ ## Training and evaluation data
35
+
36
+ More information needed
37
+
38
+ ## Training procedure
39
+
40
+ ### Training hyperparameters
41
+
42
+ The following hyperparameters were used during training:
43
+ - learning_rate: 5e-05
44
+ - train_batch_size: 128
45
+ - eval_batch_size: 128
46
+ - seed: 42
47
+ - gradient_accumulation_steps: 2
48
+ - total_train_batch_size: 256
49
+ - optimizer: Use adamw_torch with betas=(0.9,0.999) and epsilon=1e-08 and optimizer_args=No additional optimizer arguments
50
+ - lr_scheduler_type: cosine
51
+ - lr_scheduler_warmup_ratio: 0.06
52
+ - num_epochs: 30
53
+ - mixed_precision_training: Native AMP
54
+
55
+ ### Training results
56
+
57
+ | Training Loss | Epoch | Step | Validation Loss | Slot P | Slot R | Slot F1 | Slot Exact Match | Intent Acc |
58
+ |:-------------:|:-----:|:----:|:---------------:|:------:|:------:|:-------:|:----------------:|:----------:|
59
+ | No log | 1.0 | 45 | 43.3614 | 0.0 | 0.0 | 0.0 | 0.3178 | 0.0821 |
60
+ | 160.2669 | 2.0 | 90 | 27.2292 | 0.3143 | 0.2269 | 0.2635 | 0.3586 | 0.2548 |
61
+ | 66.654 | 3.0 | 135 | 19.2474 | 0.4379 | 0.4 | 0.4181 | 0.4481 | 0.4575 |
62
+ | 38.629 | 4.0 | 180 | 15.3625 | 0.4023 | 0.5408 | 0.4614 | 0.4801 | 0.5903 |
63
+ | 23.3498 | 5.0 | 225 | 12.4194 | 0.4446 | 0.5706 | 0.4998 | 0.5411 | 0.6695 |
64
+ | 12.7922 | 6.0 | 270 | 12.3227 | 0.5013 | 0.5980 | 0.5454 | 0.5691 | 0.6990 |
65
+ | 7.8613 | 7.0 | 315 | 12.8060 | 0.4926 | 0.6 | 0.5410 | 0.5642 | 0.7324 |
66
+ | 5.4037 | 8.0 | 360 | 12.9247 | 0.5086 | 0.6294 | 0.5626 | 0.5809 | 0.7388 |
67
+ | 3.6892 | 9.0 | 405 | 13.9871 | 0.5260 | 0.6343 | 0.5751 | 0.5986 | 0.7605 |
68
+ | 2.6797 | 10.0 | 450 | 14.0965 | 0.5562 | 0.6204 | 0.5865 | 0.6011 | 0.7742 |
69
+ | 2.6797 | 11.0 | 495 | 13.8520 | 0.5105 | 0.6398 | 0.5679 | 0.5775 | 0.7698 |
70
+ | 2.0031 | 12.0 | 540 | 15.0858 | 0.5491 | 0.6289 | 0.5863 | 0.6080 | 0.7698 |
71
+ | 1.3894 | 13.0 | 585 | 15.5718 | 0.5398 | 0.6408 | 0.5860 | 0.6001 | 0.7831 |
72
+
73
+
74
+ ### Framework versions
75
+
76
+ - Transformers 4.55.0
77
+ - Pytorch 2.7.0+cu126
78
+ - Datasets 3.6.0
79
+ - Tokenizers 0.21.4
intent_report_test.txt ADDED
@@ -0,0 +1,75 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ precision recall f1-score support
2
+
3
+ 0 0.88 0.88 0.88 88
4
+ 1 0.81 0.81 0.81 36
5
+ 2 0.97 0.86 0.91 35
6
+ 3 0.82 0.77 0.79 35
7
+ 4 0.96 0.85 0.90 26
8
+ 5 0.00 0.00 0.00 1
9
+ 6 0.62 0.60 0.61 43
10
+ 7 0.00 0.00 0.00 4
11
+ 8 1.00 0.78 0.88 18
12
+ 9 0.83 0.74 0.78 72
13
+ 10 0.94 0.82 0.88 39
14
+ 11 0.68 1.00 0.81 15
15
+ 12 0.39 0.56 0.46 169
16
+ 13 0.91 0.89 0.90 156
17
+ 14 0.48 0.77 0.59 13
18
+ 15 0.62 0.67 0.64 12
19
+ 16 0.68 0.77 0.72 22
20
+ 17 0.73 0.62 0.67 26
21
+ 18 0.59 0.89 0.71 27
22
+ 19 0.78 0.68 0.72 31
23
+ 20 0.78 0.71 0.74 41
24
+ 21 0.82 0.82 0.82 39
25
+ 22 0.73 0.79 0.76 124
26
+ 23 0.87 0.79 0.83 34
27
+ 24 1.00 0.80 0.89 10
28
+ 25 0.88 0.79 0.83 19
29
+ 26 0.94 0.77 0.85 57
30
+ 27 0.80 0.64 0.71 25
31
+ 28 0.00 0.00 0.00 6
32
+ 29 1.00 0.17 0.29 6
33
+ 30 0.76 0.91 0.83 67
34
+ 31 0.82 0.43 0.56 21
35
+ 32 0.66 0.65 0.65 126
36
+ 33 0.93 0.89 0.91 114
37
+ 34 0.95 0.77 0.85 26
38
+ 35 0.82 0.82 0.82 11
39
+ 36 0.82 0.78 0.80 72
40
+ 37 0.00 0.00 0.00 0
41
+ 38 0.82 0.60 0.69 15
42
+ 39 0.88 0.60 0.71 25
43
+ 40 1.00 0.84 0.91 43
44
+ 41 0.00 0.00 0.00 3
45
+ 42 0.81 0.59 0.68 51
46
+ 43 0.60 0.50 0.55 36
47
+ 44 0.96 0.84 0.90 119
48
+ 45 0.79 0.88 0.83 176
49
+ 46 0.60 0.84 0.70 32
50
+ 47 0.91 0.78 0.84 81
51
+ 48 0.87 0.80 0.84 41
52
+ 49 0.68 0.56 0.61 141
53
+ 50 0.75 0.88 0.81 209
54
+ 51 0.77 0.77 0.77 35
55
+ 52 0.83 0.90 0.86 21
56
+ 53 0.83 0.85 0.84 52
57
+ 54 0.76 0.96 0.85 23
58
+ 55 0.65 0.55 0.59 20
59
+ 56 1.00 0.94 0.97 36
60
+ 57 0.77 0.77 0.77 35
61
+ 58 0.81 0.87 0.84 63
62
+ 59 0.84 0.71 0.77 51
63
+
64
+ accuracy 0.77 2974
65
+ macro avg 0.74 0.69 0.70 2974
66
+ weighted avg 0.78 0.77 0.77 2974
67
+
68
+ Confusion matrix:
69
+ [[77 0 0 ... 0 0 0]
70
+ [ 0 29 0 ... 0 0 0]
71
+ [ 0 0 30 ... 0 0 0]
72
+ ...
73
+ [ 0 0 0 ... 27 0 0]
74
+ [ 0 0 0 ... 0 55 0]
75
+ [ 0 0 1 ... 0 0 36]]
model.safetensors CHANGED
@@ -1,3 +1,3 @@
1
  version https://git-lfs.github.com/spec/v1
2
- oid sha256:ff2ceb23a501416e10e7b17241b73b8854267231d2974775cc0f70d763224b1e
3
  size 1579896944
 
1
  version https://git-lfs.github.com/spec/v1
2
+ oid sha256:5405b4ac523b1a3fdfb6883905472c126086750b196ed444289ea8103ffcb920
3
  size 1579896944
model_predict_test.csv ADDED
The diff for this file is too large to render. See raw diff
 
slot_report_test.txt ADDED
@@ -0,0 +1,59 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ precision recall f1-score support
2
+
3
+ alarm_type 0.00 0.00 0.00 2
4
+ app_name 0.18 0.40 0.25 5
5
+ artist_name 0.49 0.64 0.55 61
6
+ audiobook_author 1.00 0.40 0.57 5
7
+ audiobook_name 0.65 0.57 0.60 23
8
+ business_name 0.56 0.64 0.60 92
9
+ business_type 0.27 0.35 0.31 31
10
+ change_amount 0.25 0.22 0.24 9
11
+ coffee_type 0.20 0.25 0.22 4
12
+ color_type 0.55 0.62 0.58 26
13
+ cooking_type 0.00 0.00 0.00 8
14
+ currency_name 0.62 0.74 0.67 50
15
+ date 0.71 0.85 0.78 415
16
+ definition_word 0.53 0.49 0.51 51
17
+ device_type 0.66 0.68 0.67 57
18
+ drink_type 0.00 0.00 0.00 1
19
+ email_address 0.60 0.67 0.63 9
20
+ email_folder 0.43 0.60 0.50 5
21
+ event_name 0.50 0.59 0.54 260
22
+ food_type 0.33 0.51 0.40 72
23
+ game_name 0.68 0.50 0.58 26
24
+ general_frequency 0.56 0.50 0.53 20
25
+ house_place 0.63 0.67 0.65 58
26
+ ingredient 0.00 0.00 0.00 6
27
+ joke_type 0.25 0.18 0.21 11
28
+ list_name 0.54 0.62 0.58 61
29
+ meal_type 0.39 0.72 0.51 18
30
+ media_type 0.64 0.80 0.71 128
31
+ movie_name 0.00 0.00 0.00 2
32
+ movie_type 0.00 0.00 0.00 3
33
+ music_album 0.00 0.00 0.00 1
34
+ music_descriptor 0.00 0.00 0.00 7
35
+ music_genre 0.67 0.76 0.71 50
36
+ news_topic 0.36 0.48 0.41 52
37
+ order_type 0.61 0.85 0.71 20
38
+ person 0.56 0.62 0.59 216
39
+ personal_info 0.31 0.57 0.40 14
40
+ place_name 0.63 0.59 0.61 281
41
+ player_setting 0.34 0.40 0.37 40
42
+ playlist_name 0.19 0.20 0.19 15
43
+ podcast_descriptor 0.37 0.29 0.33 24
44
+ podcast_name 0.25 0.35 0.29 17
45
+ radio_name 0.30 0.36 0.33 33
46
+ relation 0.36 0.44 0.40 59
47
+ song_name 0.15 0.21 0.18 39
48
+ time 0.54 0.61 0.58 191
49
+ time_zone 0.56 0.38 0.45 13
50
+ timeofday 0.67 0.67 0.67 60
51
+ transport_agency 0.80 0.89 0.84 9
52
+ transport_descriptor 0.00 0.00 0.00 2
53
+ transport_name 0.50 0.25 0.33 4
54
+ transport_type 0.73 0.80 0.76 65
55
+ weather_descriptor 0.55 0.52 0.54 82
56
+
57
+ micro avg 0.55 0.62 0.58 2813
58
+ macro avg 0.41 0.44 0.42 2813
59
+ weighted avg 0.55 0.62 0.58 2813