Hugging Face's logo

Marszzibros
/
GRPO_gemma_4b_language

Transformers
TensorBoard
Safetensors
Generated from Trainer
trl
grpo
Model card Files
xet
Metrics Community
Marszzibros commited on
Commit
ff0655d
·
verified ·
1 Parent(s): 553a4a2

Training in progress, epoch 1

Browse files
Files changed (18) hide show
  1. README.md +12 -2
  2. adapter_config.json +57 -57
  3. adapter_model.safetensors +1 -1
  4. runs/Oct10_11-48-51_raymondlab-MS-7D41/events.out.tfevents.1760111331.raymondlab-MS-7D41.2233867.0 +2 -2
  5. runs/Oct10_12-08-58_raymondlab-MS-7D41/events.out.tfevents.1760112538.raymondlab-MS-7D41.9044.0 +3 -0
  6. runs/Oct10_12-31-49_raymondlab-MS-7D41/events.out.tfevents.1760113909.raymondlab-MS-7D41.54778.0 +3 -0
  7. runs/Oct10_12-54-20_raymondlab-MS-7D41/events.out.tfevents.1760115260.raymondlab-MS-7D41.101106.0 +3 -0
  8. runs/Oct10_12-59-13_raymondlab-MS-7D41/events.out.tfevents.1760115553.raymondlab-MS-7D41.111217.0 +3 -0
  9. runs/Oct10_13-01-05_raymondlab-MS-7D41/events.out.tfevents.1760115665.raymondlab-MS-7D41.115354.0 +3 -0
  10. runs/Oct10_13-05-11_raymondlab-MS-7D41/events.out.tfevents.1760115911.raymondlab-MS-7D41.124117.0 +3 -0
  11. runs/Oct10_13-07-12_raymondlab-MS-7D41/events.out.tfevents.1760116032.raymondlab-MS-7D41.127767.0 +3 -0
  12. runs/Oct10_13-12-35_raymondlab-MS-7D41/events.out.tfevents.1760116356.raymondlab-MS-7D41.139454.0 +3 -0
  13. runs/Oct10_13-20-50_raymondlab-MS-7D41/events.out.tfevents.1760116851.raymondlab-MS-7D41.156749.0 +3 -0
  14. runs/Oct10_13-23-20_raymondlab-MS-7D41/events.out.tfevents.1760117000.raymondlab-MS-7D41.162183.0 +3 -0
  15. runs/Oct10_13-24-56_raymondlab-MS-7D41/events.out.tfevents.1760117096.raymondlab-MS-7D41.166138.0 +3 -0
  16. runs/Oct10_13-26-56_raymondlab-MS-7D41/events.out.tfevents.1760117217.raymondlab-MS-7D41.170464.0 +3 -0
  17. runs/Oct10_13-27-59_raymondlab-MS-7D41/events.out.tfevents.1760117279.raymondlab-MS-7D41.173037.0 +3 -0
  18. training_args.bin +2 -2
README.md CHANGED
@@ -4,8 +4,8 @@ library_name: transformers
4
  model_name: GRPO_gemma_4b_language
5
  tags:
6
  - generated_from_trainer
7
- - sft
8
  - trl
 
9
  licence: license
10
  ---
11
 
@@ -30,7 +30,7 @@ print(output["generated_text"])
30
 
31
 
32
 
33
- This model was trained with SFT.
34
 
35
  ### Framework versions
36
 
@@ -42,7 +42,17 @@ This model was trained with SFT.
42
 
43
  ## Citations
44
 
 
45
 
 
 
 
 
 
 
 
 
 
46
 
47
  Cite TRL as:
48
 
 
4
  model_name: GRPO_gemma_4b_language
5
  tags:
6
  - generated_from_trainer
 
7
  - trl
8
+ - grpo
9
  licence: license
10
  ---
11
 
 
30
 
31
 
32
 
33
+ This model was trained with GRPO, a method introduced in [DeepSeekMath: Pushing the Limits of Mathematical Reasoning in Open Language Models](https://huggingface.co/papers/2402.03300).
34
 
35
  ### Framework versions
36
 
 
42
 
43
  ## Citations
44
 
45
+ Cite GRPO as:
46
 
47
+ ```bibtex
48
+ @article{shao2024deepseekmath,
49
+ title = {{DeepSeekMath: Pushing the Limits of Mathematical Reasoning in Open Language Models}},
50
+ author = {Zhihong Shao and Peiyi Wang and Qihao Zhu and Runxin Xu and Junxiao Song and Mingchuan Zhang and Y. K. Li and Y. Wu and Daya Guo},
51
+ year = 2024,
52
+ eprint = {arXiv:2402.03300},
53
+ }
54
+
55
+ ```
56
 
57
  Cite TRL as:
58
 
adapter_config.json CHANGED
@@ -25,74 +25,74 @@
25
  "rank_pattern": {},
26
  "revision": null,
27
  "target_modules": [
28
- "language_model.layers.2.self_attn.k_proj",
29
- "language_model.layers.16.self_attn.q_proj",
30
- "language_model.layers.3.self_attn.q_proj",
31
- "language_model.layers.22.self_attn.q_proj",
32
- "language_model.layers.12.self_attn.k_proj",
33
- "language_model.layers.0.self_attn.k_proj",
34
  "language_model.layers.5.self_attn.q_proj",
35
- "language_model.layers.31.self_attn.q_proj",
36
- "language_model.layers.0.self_attn.q_proj",
37
- "language_model.layers.7.self_attn.k_proj",
38
- "language_model.layers.9.self_attn.k_proj",
39
- "language_model.layers.3.self_attn.k_proj",
40
- "language_model.layers.13.self_attn.q_proj",
41
- "language_model.layers.23.self_attn.k_proj",
42
- "language_model.layers.12.self_attn.q_proj",
43
- "language_model.layers.20.self_attn.k_proj",
44
- "language_model.layers.24.self_attn.k_proj",
45
- "language_model.layers.8.self_attn.q_proj",
46
- "language_model.layers.23.self_attn.q_proj",
47
- "language_model.layers.14.self_attn.k_proj",
48
- "language_model.layers.15.self_attn.k_proj",
49
- "language_model.layers.10.self_attn.k_proj",
50
  "language_model.layers.26.self_attn.k_proj",
 
 
51
  "language_model.layers.4.self_attn.k_proj",
52
- "language_model.layers.11.self_attn.q_proj",
53
- "language_model.layers.13.self_attn.k_proj",
54
- "language_model.layers.18.self_attn.q_proj",
55
- "language_model.layers.29.self_attn.q_proj",
56
- "language_model.layers.27.self_attn.k_proj",
57
- "language_model.layers.30.self_attn.q_proj",
58
- "language_model.layers.8.self_attn.k_proj",
59
- "language_model.layers.31.self_attn.k_proj",
60
  "language_model.layers.28.self_attn.k_proj",
61
- "language_model.layers.32.self_attn.q_proj",
62
- "language_model.layers.18.self_attn.k_proj",
63
- "language_model.layers.20.self_attn.q_proj",
64
- "language_model.layers.30.self_attn.k_proj",
65
- "language_model.layers.21.self_attn.q_proj",
66
- "language_model.layers.7.self_attn.q_proj",
67
- "language_model.layers.19.self_attn.k_proj",
68
- "language_model.layers.1.self_attn.q_proj",
69
  "language_model.layers.5.self_attn.k_proj",
70
- "language_model.layers.25.self_attn.k_proj",
71
- "language_model.layers.33.self_attn.q_proj",
72
- "language_model.layers.6.self_attn.q_proj",
73
- "language_model.layers.2.self_attn.q_proj",
74
  "language_model.layers.14.self_attn.q_proj",
75
- "language_model.layers.24.self_attn.q_proj",
76
- "language_model.layers.22.self_attn.k_proj",
77
- "language_model.layers.4.self_attn.q_proj",
 
 
 
 
 
 
78
  "language_model.layers.27.self_attn.q_proj",
79
- "language_model.layers.19.self_attn.q_proj",
80
- "language_model.layers.10.self_attn.q_proj",
81
- "language_model.layers.17.self_attn.q_proj",
82
- "language_model.layers.15.self_attn.q_proj",
83
- "language_model.layers.16.self_attn.k_proj",
84
- "language_model.layers.11.self_attn.k_proj",
85
- "language_model.layers.17.self_attn.k_proj",
86
  "language_model.layers.25.self_attn.q_proj",
87
- "language_model.layers.32.self_attn.k_proj",
88
- "language_model.layers.28.self_attn.q_proj",
89
- "language_model.layers.9.self_attn.q_proj",
90
- "language_model.layers.21.self_attn.k_proj",
91
  "language_model.layers.26.self_attn.q_proj",
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
92
  "language_model.layers.33.self_attn.k_proj",
93
- "language_model.layers.1.self_attn.k_proj",
94
  "language_model.layers.6.self_attn.k_proj",
95
- "language_model.layers.29.self_attn.k_proj"
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
96
  ],
97
  "target_parameters": null,
98
  "task_type": "CAUSAL_LM",
 
25
  "rank_pattern": {},
26
  "revision": null,
27
  "target_modules": [
28
+ "language_model.layers.6.self_attn.q_proj",
29
+ "language_model.layers.18.self_attn.k_proj",
 
 
 
 
30
  "language_model.layers.5.self_attn.q_proj",
31
+ "language_model.layers.17.self_attn.q_proj",
32
+ "language_model.layers.1.self_attn.q_proj",
33
+ "language_model.layers.21.self_attn.k_proj",
 
 
 
 
 
 
 
 
 
 
 
 
34
  "language_model.layers.26.self_attn.k_proj",
35
+ "language_model.layers.16.self_attn.q_proj",
36
+ "language_model.layers.10.self_attn.q_proj",
37
  "language_model.layers.4.self_attn.k_proj",
38
+ "language_model.layers.20.self_attn.k_proj",
 
 
 
 
 
 
 
39
  "language_model.layers.28.self_attn.k_proj",
40
+ "language_model.layers.23.self_attn.k_proj",
 
 
 
 
 
 
 
41
  "language_model.layers.5.self_attn.k_proj",
42
+ "language_model.layers.13.self_attn.q_proj",
 
 
 
43
  "language_model.layers.14.self_attn.q_proj",
44
+ "language_model.layers.9.self_attn.k_proj",
45
+ "language_model.layers.33.self_attn.q_proj",
46
+ "language_model.layers.1.self_attn.k_proj",
47
+ "language_model.layers.3.self_attn.k_proj",
48
+ "language_model.layers.13.self_attn.k_proj",
49
+ "language_model.layers.22.self_attn.q_proj",
50
+ "language_model.layers.7.self_attn.q_proj",
51
+ "language_model.layers.2.self_attn.k_proj",
52
+ "language_model.layers.14.self_attn.k_proj",
53
  "language_model.layers.27.self_attn.q_proj",
54
+ "language_model.layers.30.self_attn.q_proj",
55
+ "language_model.layers.21.self_attn.q_proj",
56
+ "language_model.layers.24.self_attn.q_proj",
 
 
 
 
57
  "language_model.layers.25.self_attn.q_proj",
58
+ "language_model.layers.8.self_attn.k_proj",
59
+ "language_model.layers.11.self_attn.q_proj",
 
 
60
  "language_model.layers.26.self_attn.q_proj",
61
+ "language_model.layers.20.self_attn.q_proj",
62
+ "language_model.layers.0.self_attn.k_proj",
63
+ "language_model.layers.0.self_attn.q_proj",
64
+ "language_model.layers.4.self_attn.q_proj",
65
+ "language_model.layers.29.self_attn.q_proj",
66
+ "language_model.layers.32.self_attn.q_proj",
67
+ "language_model.layers.28.self_attn.q_proj",
68
+ "language_model.layers.17.self_attn.k_proj",
69
+ "language_model.layers.10.self_attn.k_proj",
70
+ "language_model.layers.15.self_attn.q_proj",
71
+ "language_model.layers.30.self_attn.k_proj",
72
+ "language_model.layers.24.self_attn.k_proj",
73
+ "language_model.layers.22.self_attn.k_proj",
74
+ "language_model.layers.8.self_attn.q_proj",
75
+ "language_model.layers.27.self_attn.k_proj",
76
+ "language_model.layers.31.self_attn.q_proj",
77
  "language_model.layers.33.self_attn.k_proj",
78
+ "language_model.layers.3.self_attn.q_proj",
79
  "language_model.layers.6.self_attn.k_proj",
80
+ "language_model.layers.11.self_attn.k_proj",
81
+ "language_model.layers.12.self_attn.q_proj",
82
+ "language_model.layers.29.self_attn.k_proj",
83
+ "language_model.layers.18.self_attn.q_proj",
84
+ "language_model.layers.2.self_attn.q_proj",
85
+ "language_model.layers.15.self_attn.k_proj",
86
+ "language_model.layers.19.self_attn.k_proj",
87
+ "language_model.layers.7.self_attn.k_proj",
88
+ "language_model.layers.9.self_attn.q_proj",
89
+ "language_model.layers.23.self_attn.q_proj",
90
+ "language_model.layers.25.self_attn.k_proj",
91
+ "language_model.layers.31.self_attn.k_proj",
92
+ "language_model.layers.32.self_attn.k_proj",
93
+ "language_model.layers.12.self_attn.k_proj",
94
+ "language_model.layers.16.self_attn.k_proj",
95
+ "language_model.layers.19.self_attn.q_proj"
96
  ],
97
  "target_parameters": null,
98
  "task_type": "CAUSAL_LM",
adapter_model.safetensors CHANGED
@@ -1,3 +1,3 @@
1
  version https://git-lfs.github.com/spec/v1
2
- oid sha256:81bdd65868be01652215114e4f52a3bbb31e8a90bb86fdbee66ce1473917cd1f
3
  size 17846024
 
1
  version https://git-lfs.github.com/spec/v1
2
+ oid sha256:ac5f216817076ff6dad7650755d10457c8fc453a8fafe27cdeeb3d6b1b9c6d4f
3
  size 17846024
runs/Oct10_11-48-51_raymondlab-MS-7D41/events.out.tfevents.1760111331.raymondlab-MS-7D41.2233867.0 CHANGED
@@ -1,3 +1,3 @@
1
  version https://git-lfs.github.com/spec/v1
2
- oid sha256:de3d85420e009bfcbc88f4a8721e287e45850465977c460675a1599b26f4e372
3
- size 10106
 
1
  version https://git-lfs.github.com/spec/v1
2
+ oid sha256:404ef7e98d56ae4d37aeabd6a26b224a5b7f7d348925ce7a7cbb4c3efe9bfbaa
3
+ size 11606
runs/Oct10_12-08-58_raymondlab-MS-7D41/events.out.tfevents.1760112538.raymondlab-MS-7D41.9044.0 ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:d264b42cf5bd9a4a597d917bf4ea7516cf274c56239bc90d44098858c9dc02da
3
+ size 15893
runs/Oct10_12-31-49_raymondlab-MS-7D41/events.out.tfevents.1760113909.raymondlab-MS-7D41.54778.0 ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:cf8599720c269e0ece809cab281e1164d2aa2a88b43e1adcbbd40f99c9e85a85
3
+ size 8778
runs/Oct10_12-54-20_raymondlab-MS-7D41/events.out.tfevents.1760115260.raymondlab-MS-7D41.101106.0 ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:d23a5eeec31f07a973595f353d51bb89c2e7ea9a47ddbd25d1d31dccceb13c01
3
+ size 10201
runs/Oct10_12-59-13_raymondlab-MS-7D41/events.out.tfevents.1760115553.raymondlab-MS-7D41.111217.0 ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:d04cfb7df79e65531ebb43dae81507cb551f7cf4500a1fdec72454ece3c3dc1a
3
+ size 8777
runs/Oct10_13-01-05_raymondlab-MS-7D41/events.out.tfevents.1760115665.raymondlab-MS-7D41.115354.0 ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:2aad985f62ea6374d8c6f9f9cec567b537a730a11b8622e6883674dfa68347a0
3
+ size 10200
runs/Oct10_13-05-11_raymondlab-MS-7D41/events.out.tfevents.1760115911.raymondlab-MS-7D41.124117.0 ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:2d6bb2c9f63b02160c5889d403fa26a2801e85fd7123f938784fb569d53d1dcc
3
+ size 8777
runs/Oct10_13-07-12_raymondlab-MS-7D41/events.out.tfevents.1760116032.raymondlab-MS-7D41.127767.0 ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:25b4c42d5c4f03c692c6917d1bdc4b548350c7d46fa9fc61e1e2dcfd699c4c7b
3
+ size 8777
runs/Oct10_13-12-35_raymondlab-MS-7D41/events.out.tfevents.1760116356.raymondlab-MS-7D41.139454.0 ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:666a27afb72b83149f24edb3d9f015e88e48c296a7c33286be48b68a33895269
3
+ size 11623
runs/Oct10_13-20-50_raymondlab-MS-7D41/events.out.tfevents.1760116851.raymondlab-MS-7D41.156749.0 ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:a6214b8f455ab125acb5d6c98858b9f2dc15e02c2954226202eafc624e1023b5
3
+ size 8777
runs/Oct10_13-23-20_raymondlab-MS-7D41/events.out.tfevents.1760117000.raymondlab-MS-7D41.162183.0 ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:fecf9d2a0b7c1b712635bbc1dcee5045e2ac19c25d2fa4000e6dadf2f13be969
3
+ size 8777
runs/Oct10_13-24-56_raymondlab-MS-7D41/events.out.tfevents.1760117096.raymondlab-MS-7D41.166138.0 ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:eb0606724e8e6dbaaaa63216953dab76d72977b4174538bd1b94f538bd9f2a22
3
+ size 8777
runs/Oct10_13-26-56_raymondlab-MS-7D41/events.out.tfevents.1760117217.raymondlab-MS-7D41.170464.0 ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:aa11975c63ddf28cac27744cbfc0275ecd67cadb174df45dbedd3fc067fce448
3
+ size 8777
runs/Oct10_13-27-59_raymondlab-MS-7D41/events.out.tfevents.1760117279.raymondlab-MS-7D41.173037.0 ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:31bc93fa04c55907c5afe96d3154836c1a9cd7db4c238c51ef621d20ce8f2c87
3
+ size 45798
training_args.bin CHANGED
@@ -1,3 +1,3 @@
1
  version https://git-lfs.github.com/spec/v1
2
- oid sha256:8f3bbc87383d8a8bfebf51cb1193e609256a288e3d9e21820a4221175627b9f2
3
- size 6225
 
1
  version https://git-lfs.github.com/spec/v1
2
+ oid sha256:15df0ddeb6ae0d6e0ab9f610669db79a69eef0abc397f317c3b45292f9caf8e8
3
+ size 7185