Commit
·
245ff98
1
Parent(s):
0fc1eb6
Update README.md
Browse files
README.md
CHANGED
|
@@ -10,9 +10,11 @@ license: apache-2.0
|
|
| 10 |
- 採用8bits精度載入模型權重 (load_in_8bit=True)
|
| 11 |
- 使用peft library的LoRA訓練方式，進行fine-tuning:
|
| 12 |
|
|
|
|
| 13 |
lora_alpha = 16
|
| 14 |
lora_dropout = 0.1
|
| 15 |
lora_r = 8
|
|
|
|
| 16 |
|
| 17 |
### 資料集
|
| 18 |
(語料包括:英,中,日,韓) HF連結: https://huggingface.co/datasets/timdettmers/openassistant-guanaco
|
|
|
|
| 10 |
- 採用8bits精度載入模型權重 (load_in_8bit=True)
|
| 11 |
- 使用peft library的LoRA訓練方式，進行fine-tuning:
|
| 12 |
|
| 13 |
+
```
|
| 14 |
lora_alpha = 16
|
| 15 |
lora_dropout = 0.1
|
| 16 |
lora_r = 8
|
| 17 |
+
```
|
| 18 |
|
| 19 |
### 資料集
|
| 20 |
(語料包括:英,中,日,韓) HF連結: https://huggingface.co/datasets/timdettmers/openassistant-guanaco
|