Files changed (1) hide show
  1. README.md +215 -203
README.md CHANGED
@@ -1,203 +1,215 @@
1
- ---
2
- language:
3
- - en
4
- license: apache-2.0
5
- library_name: transformers
6
- tags:
7
- - role-play
8
- - fine-tuned
9
- - qwen2.5
10
- base_model:
11
- - Qwen/Qwen2.5-14B-Instruct
12
- pipeline_tag: text-generation
13
- model-index:
14
- - name: oxy-1-small
15
- results:
16
- - task:
17
- type: text-generation
18
- name: Text Generation
19
- dataset:
20
- name: IFEval (0-Shot)
21
- type: HuggingFaceH4/ifeval
22
- args:
23
- num_few_shot: 0
24
- metrics:
25
- - type: inst_level_strict_acc and prompt_level_strict_acc
26
- value: 62.45
27
- name: strict accuracy
28
- source:
29
- url: https://huggingface.co/spaces/open-llm-leaderboard/open_llm_leaderboard?query=oxyapi/oxy-1-small
30
- name: Open LLM Leaderboard
31
- - task:
32
- type: text-generation
33
- name: Text Generation
34
- dataset:
35
- name: BBH (3-Shot)
36
- type: BBH
37
- args:
38
- num_few_shot: 3
39
- metrics:
40
- - type: acc_norm
41
- value: 41.18
42
- name: normalized accuracy
43
- source:
44
- url: https://huggingface.co/spaces/open-llm-leaderboard/open_llm_leaderboard?query=oxyapi/oxy-1-small
45
- name: Open LLM Leaderboard
46
- - task:
47
- type: text-generation
48
- name: Text Generation
49
- dataset:
50
- name: MATH Lvl 5 (4-Shot)
51
- type: hendrycks/competition_math
52
- args:
53
- num_few_shot: 4
54
- metrics:
55
- - type: exact_match
56
- value: 18.28
57
- name: exact match
58
- source:
59
- url: https://huggingface.co/spaces/open-llm-leaderboard/open_llm_leaderboard?query=oxyapi/oxy-1-small
60
- name: Open LLM Leaderboard
61
- - task:
62
- type: text-generation
63
- name: Text Generation
64
- dataset:
65
- name: GPQA (0-shot)
66
- type: Idavidrein/gpqa
67
- args:
68
- num_few_shot: 0
69
- metrics:
70
- - type: acc_norm
71
- value: 16.22
72
- name: acc_norm
73
- source:
74
- url: https://huggingface.co/spaces/open-llm-leaderboard/open_llm_leaderboard?query=oxyapi/oxy-1-small
75
- name: Open LLM Leaderboard
76
- - task:
77
- type: text-generation
78
- name: Text Generation
79
- dataset:
80
- name: MuSR (0-shot)
81
- type: TAUR-Lab/MuSR
82
- args:
83
- num_few_shot: 0
84
- metrics:
85
- - type: acc_norm
86
- value: 16.28
87
- name: acc_norm
88
- source:
89
- url: https://huggingface.co/spaces/open-llm-leaderboard/open_llm_leaderboard?query=oxyapi/oxy-1-small
90
- name: Open LLM Leaderboard
91
- - task:
92
- type: text-generation
93
- name: Text Generation
94
- dataset:
95
- name: MMLU-PRO (5-shot)
96
- type: TIGER-Lab/MMLU-Pro
97
- config: main
98
- split: test
99
- args:
100
- num_few_shot: 5
101
- metrics:
102
- - type: acc
103
- value: 44.45
104
- name: accuracy
105
- source:
106
- url: https://huggingface.co/spaces/open-llm-leaderboard/open_llm_leaderboard?query=oxyapi/oxy-1-small
107
- name: Open LLM Leaderboard
108
- ---
109
- ### exl2 quant (measurement.json in main branch)
110
- ---
111
- ### check revisions for quants
112
- ---
113
-
114
-
115
- ![Oxy 1 Small](https://cdn-uploads.huggingface.co/production/uploads/64fb80c8bb362cbf2ff96c7e/tTIVIblPUbTYnlvHQQjXB.png)
116
-
117
- ## Introduction
118
-
119
- **Oxy 1 Small** is a fine-tuned version of the [Qwen/Qwen2.5-14B-Instruct](https://huggingface.co/Qwen/Qwen/Qwen2.5-14B-Instruct) language model, specialized for **role-play** scenarios. Despite its small size, it delivers impressive performance in generating engaging dialogues and interactive storytelling.
120
-
121
- Developed by **Oxygen (oxyapi)**, with contributions from **TornadoSoftwares**, Oxy 1 Small aims to provide an accessible and efficient language model for creative and immersive role-play experiences.
122
-
123
- ## Model Details
124
-
125
- - **Model Name**: Oxy 1 Small
126
- - **Model ID**: [oxyapi/oxy-1-small](https://huggingface.co/oxyapi/oxy-1-small)
127
- - **Base Model**: [Qwen/Qwen2.5-14B-Instruct](https://huggingface.co/Qwen/Qwen2.5-14B-Instruct)
128
- - **Model Type**: Chat Completions
129
- - **Prompt Format**: ChatML
130
- - **License**: Apache-2.0
131
- - **Language**: English
132
- - **Tokenizer**: [Qwen/Qwen2.5-14B-Instruct](https://huggingface.co/Qwen/Qwen2.5-14B-Instruct)
133
- - **Max Input Tokens**: 32,768
134
- - **Max Output Tokens**: 8,192
135
-
136
- ### Features
137
-
138
- - **Fine-tuned for Role-Play**: Specially trained to generate dynamic and contextually rich role-play dialogues.
139
- - **Efficient**: Compact model size allows for faster inference and reduced computational resources.
140
- - **Parameter Support**:
141
- - `temperature`
142
- - `top_p`
143
- - `top_k`
144
- - `frequency_penalty`
145
- - `presence_penalty`
146
- - `max_tokens`
147
-
148
- ### Metadata
149
-
150
- - **Owned by**: Oxygen (oxyapi)
151
- - **Contributors**: TornadoSoftwares
152
- - **Description**: A Qwen/Qwen2.5-14B-Instruct fine-tune for role-play trained on custom datasets
153
-
154
- ## Usage
155
-
156
- To utilize Oxy 1 Small for text generation in role-play scenarios, you can load the model using the Hugging Face Transformers library:
157
-
158
- ```python
159
- from transformers import AutoModelForCausalLM, AutoTokenizer
160
-
161
- tokenizer = AutoTokenizer.from_pretrained("oxyapi/oxy-1-small")
162
- model = AutoModelForCausalLM.from_pretrained("oxyapi/oxy-1-small")
163
-
164
- prompt = "You are a wise old wizard in a mystical land. A traveler approaches you seeking advice."
165
- inputs = tokenizer(prompt, return_tensors="pt")
166
- outputs = model.generate(**inputs, max_length=500)
167
- response = tokenizer.decode(outputs[0], skip_special_tokens=True)
168
- print(response)
169
- ```
170
-
171
- ## Performance
172
-
173
- Performance benchmarks for Oxy 1 Small are not available at this time. Future updates may include detailed evaluations on relevant datasets.
174
-
175
- ## License
176
-
177
- This model is licensed under the [Apache 2.0 License](https://www.apache.org/licenses/LICENSE-2.0).
178
-
179
- ## Citation
180
-
181
- If you find Oxy 1 Small useful in your research or applications, please cite it as:
182
-
183
- ```
184
- @misc{oxy1small2024,
185
- title={Oxy 1 Small: A Fine-Tuned Qwen2.5-14B-Instruct Model for Role-Play},
186
- author={Oxygen (oxyapi)},
187
- year={2024},
188
- howpublished={\url{https://huggingface.co/oxyapi/oxy-1-small}},
189
- }
190
- ```
191
- # [Open LLM Leaderboard Evaluation Results](https://huggingface.co/spaces/open-llm-leaderboard/open_llm_leaderboard)
192
- Detailed results can be found [here](https://huggingface.co/datasets/open-llm-leaderboard/details_oxyapi__oxy-1-small)
193
-
194
- | Metric |Value|
195
- |-------------------|----:|
196
- |Avg. |33.14|
197
- |IFEval (0-Shot) |62.45|
198
- |BBH (3-Shot) |41.18|
199
- |MATH Lvl 5 (4-Shot)|18.28|
200
- |GPQA (0-shot) |16.22|
201
- |MuSR (0-shot) |16.28|
202
- |MMLU-PRO (5-shot) |44.45|
203
-
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ ---
2
+ language:
3
+ - zho
4
+ - eng
5
+ - fra
6
+ - spa
7
+ - por
8
+ - deu
9
+ - ita
10
+ - rus
11
+ - jpn
12
+ - kor
13
+ - vie
14
+ - tha
15
+ - ara
16
+ license: apache-2.0
17
+ library_name: transformers
18
+ tags:
19
+ - role-play
20
+ - fine-tuned
21
+ - qwen2.5
22
+ base_model:
23
+ - Qwen/Qwen2.5-14B-Instruct
24
+ pipeline_tag: text-generation
25
+ model-index:
26
+ - name: oxy-1-small
27
+ results:
28
+ - task:
29
+ type: text-generation
30
+ name: Text Generation
31
+ dataset:
32
+ name: IFEval (0-Shot)
33
+ type: HuggingFaceH4/ifeval
34
+ args:
35
+ num_few_shot: 0
36
+ metrics:
37
+ - type: inst_level_strict_acc and prompt_level_strict_acc
38
+ value: 62.45
39
+ name: strict accuracy
40
+ source:
41
+ url: https://huggingface.co/spaces/open-llm-leaderboard/open_llm_leaderboard?query=oxyapi/oxy-1-small
42
+ name: Open LLM Leaderboard
43
+ - task:
44
+ type: text-generation
45
+ name: Text Generation
46
+ dataset:
47
+ name: BBH (3-Shot)
48
+ type: BBH
49
+ args:
50
+ num_few_shot: 3
51
+ metrics:
52
+ - type: acc_norm
53
+ value: 41.18
54
+ name: normalized accuracy
55
+ source:
56
+ url: https://huggingface.co/spaces/open-llm-leaderboard/open_llm_leaderboard?query=oxyapi/oxy-1-small
57
+ name: Open LLM Leaderboard
58
+ - task:
59
+ type: text-generation
60
+ name: Text Generation
61
+ dataset:
62
+ name: MATH Lvl 5 (4-Shot)
63
+ type: hendrycks/competition_math
64
+ args:
65
+ num_few_shot: 4
66
+ metrics:
67
+ - type: exact_match
68
+ value: 18.28
69
+ name: exact match
70
+ source:
71
+ url: https://huggingface.co/spaces/open-llm-leaderboard/open_llm_leaderboard?query=oxyapi/oxy-1-small
72
+ name: Open LLM Leaderboard
73
+ - task:
74
+ type: text-generation
75
+ name: Text Generation
76
+ dataset:
77
+ name: GPQA (0-shot)
78
+ type: Idavidrein/gpqa
79
+ args:
80
+ num_few_shot: 0
81
+ metrics:
82
+ - type: acc_norm
83
+ value: 16.22
84
+ name: acc_norm
85
+ source:
86
+ url: https://huggingface.co/spaces/open-llm-leaderboard/open_llm_leaderboard?query=oxyapi/oxy-1-small
87
+ name: Open LLM Leaderboard
88
+ - task:
89
+ type: text-generation
90
+ name: Text Generation
91
+ dataset:
92
+ name: MuSR (0-shot)
93
+ type: TAUR-Lab/MuSR
94
+ args:
95
+ num_few_shot: 0
96
+ metrics:
97
+ - type: acc_norm
98
+ value: 16.28
99
+ name: acc_norm
100
+ source:
101
+ url: https://huggingface.co/spaces/open-llm-leaderboard/open_llm_leaderboard?query=oxyapi/oxy-1-small
102
+ name: Open LLM Leaderboard
103
+ - task:
104
+ type: text-generation
105
+ name: Text Generation
106
+ dataset:
107
+ name: MMLU-PRO (5-shot)
108
+ type: TIGER-Lab/MMLU-Pro
109
+ config: main
110
+ split: test
111
+ args:
112
+ num_few_shot: 5
113
+ metrics:
114
+ - type: acc
115
+ value: 44.45
116
+ name: accuracy
117
+ source:
118
+ url: https://huggingface.co/spaces/open-llm-leaderboard/open_llm_leaderboard?query=oxyapi/oxy-1-small
119
+ name: Open LLM Leaderboard
120
+ ---
121
+ ### exl2 quant (measurement.json in main branch)
122
+ ---
123
+ ### check revisions for quants
124
+ ---
125
+
126
+
127
+ ![Oxy 1 Small](https://cdn-uploads.huggingface.co/production/uploads/64fb80c8bb362cbf2ff96c7e/tTIVIblPUbTYnlvHQQjXB.png)
128
+
129
+ ## Introduction
130
+
131
+ **Oxy 1 Small** is a fine-tuned version of the [Qwen/Qwen2.5-14B-Instruct](https://huggingface.co/Qwen/Qwen/Qwen2.5-14B-Instruct) language model, specialized for **role-play** scenarios. Despite its small size, it delivers impressive performance in generating engaging dialogues and interactive storytelling.
132
+
133
+ Developed by **Oxygen (oxyapi)**, with contributions from **TornadoSoftwares**, Oxy 1 Small aims to provide an accessible and efficient language model for creative and immersive role-play experiences.
134
+
135
+ ## Model Details
136
+
137
+ - **Model Name**: Oxy 1 Small
138
+ - **Model ID**: [oxyapi/oxy-1-small](https://huggingface.co/oxyapi/oxy-1-small)
139
+ - **Base Model**: [Qwen/Qwen2.5-14B-Instruct](https://huggingface.co/Qwen/Qwen2.5-14B-Instruct)
140
+ - **Model Type**: Chat Completions
141
+ - **Prompt Format**: ChatML
142
+ - **License**: Apache-2.0
143
+ - **Language**: English
144
+ - **Tokenizer**: [Qwen/Qwen2.5-14B-Instruct](https://huggingface.co/Qwen/Qwen2.5-14B-Instruct)
145
+ - **Max Input Tokens**: 32,768
146
+ - **Max Output Tokens**: 8,192
147
+
148
+ ### Features
149
+
150
+ - **Fine-tuned for Role-Play**: Specially trained to generate dynamic and contextually rich role-play dialogues.
151
+ - **Efficient**: Compact model size allows for faster inference and reduced computational resources.
152
+ - **Parameter Support**:
153
+ - `temperature`
154
+ - `top_p`
155
+ - `top_k`
156
+ - `frequency_penalty`
157
+ - `presence_penalty`
158
+ - `max_tokens`
159
+
160
+ ### Metadata
161
+
162
+ - **Owned by**: Oxygen (oxyapi)
163
+ - **Contributors**: TornadoSoftwares
164
+ - **Description**: A Qwen/Qwen2.5-14B-Instruct fine-tune for role-play trained on custom datasets
165
+
166
+ ## Usage
167
+
168
+ To utilize Oxy 1 Small for text generation in role-play scenarios, you can load the model using the Hugging Face Transformers library:
169
+
170
+ ```python
171
+ from transformers import AutoModelForCausalLM, AutoTokenizer
172
+
173
+ tokenizer = AutoTokenizer.from_pretrained("oxyapi/oxy-1-small")
174
+ model = AutoModelForCausalLM.from_pretrained("oxyapi/oxy-1-small")
175
+
176
+ prompt = "You are a wise old wizard in a mystical land. A traveler approaches you seeking advice."
177
+ inputs = tokenizer(prompt, return_tensors="pt")
178
+ outputs = model.generate(**inputs, max_length=500)
179
+ response = tokenizer.decode(outputs[0], skip_special_tokens=True)
180
+ print(response)
181
+ ```
182
+
183
+ ## Performance
184
+
185
+ Performance benchmarks for Oxy 1 Small are not available at this time. Future updates may include detailed evaluations on relevant datasets.
186
+
187
+ ## License
188
+
189
+ This model is licensed under the [Apache 2.0 License](https://www.apache.org/licenses/LICENSE-2.0).
190
+
191
+ ## Citation
192
+
193
+ If you find Oxy 1 Small useful in your research or applications, please cite it as:
194
+
195
+ ```
196
+ @misc{oxy1small2024,
197
+ title={Oxy 1 Small: A Fine-Tuned Qwen2.5-14B-Instruct Model for Role-Play},
198
+ author={Oxygen (oxyapi)},
199
+ year={2024},
200
+ howpublished={\url{https://huggingface.co/oxyapi/oxy-1-small}},
201
+ }
202
+ ```
203
+ # [Open LLM Leaderboard Evaluation Results](https://huggingface.co/spaces/open-llm-leaderboard/open_llm_leaderboard)
204
+ Detailed results can be found [here](https://huggingface.co/datasets/open-llm-leaderboard/details_oxyapi__oxy-1-small)
205
+
206
+ | Metric |Value|
207
+ |-------------------|----:|
208
+ |Avg. |33.14|
209
+ |IFEval (0-Shot) |62.45|
210
+ |BBH (3-Shot) |41.18|
211
+ |MATH Lvl 5 (4-Shot)|18.28|
212
+ |GPQA (0-shot) |16.22|
213
+ |MuSR (0-shot) |16.28|
214
+ |MMLU-PRO (5-shot) |44.45|
215
+