Commit
·
0fc1eb6
1
Parent(s):
d7f4da9
Update README.md
Browse files
README.md
CHANGED
|
@@ -9,7 +9,13 @@ license: apache-2.0
|
|
| 9 |
- 基礎模型: huggingface平台的Llama-2-7b (https://huggingface.co/meta-llama/Llama-2-7b-hf)
|
| 10 |
- 採用8bits精度載入模型權重 (load_in_8bit=True)
|
| 11 |
- 使用peft library的LoRA訓練方式，進行fine-tuning:
|
|
|
|
| 12 |
lora_alpha = 16
|
| 13 |
lora_dropout = 0.1
|
| 14 |
lora_r = 8
|
| 15 |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 9 |
- 基礎模型: huggingface平台的Llama-2-7b (https://huggingface.co/meta-llama/Llama-2-7b-hf)
|
| 10 |
- 採用8bits精度載入模型權重 (load_in_8bit=True)
|
| 11 |
- 使用peft library的LoRA訓練方式，進行fine-tuning:
|
| 12 |
+
|
| 13 |
lora_alpha = 16
|
| 14 |
lora_dropout = 0.1
|
| 15 |
lora_r = 8
|
| 16 |
|
| 17 |
+
### 資料集
|
| 18 |
+
(語料包括:英,中,日,韓) HF連結: https://huggingface.co/datasets/timdettmers/openassistant-guanaco
|
| 19 |
+
|
| 20 |
+
### 訓練環境
|
| 21 |
+
使用google colab 免費資源(GPU: T4, 15GB)
|