Update README.md
Browse files
README.md
CHANGED
|
@@ -10,6 +10,10 @@
|
|
| 10 |
|
| 11 |
This repository features LLaVA v1.5 trained with the Meta-Llama-3-8B-Instruct LLM. This integration aims to leverage the strengths of both models to offer advanced vision-language understanding.
|
| 12 |
|
|
|
|
|
|
|
|
|
|
|
|
|
| 13 |
|
| 14 |
## Key Components
|
| 15 |
|
|
|
|
| 10 |
|
| 11 |
This repository features LLaVA v1.5 trained with the Meta-Llama-3-8B-Instruct LLM. This integration aims to leverage the strengths of both models to offer advanced vision-language understanding.
|
| 12 |
|
| 13 |
+
## Training Strategy
|
| 14 |
+
- **Pretraining:** Only Vision-to-Language projector is trained. The rest of the model is frozen.
|
| 15 |
+
- **Fine-tuning:** LLM is LoRA fine-tuned. Only the vision-backbone (CLIP) is kept frozen.
|
| 16 |
+
- **Note:** The repository contains projector and LoRA weights.
|
| 17 |
|
| 18 |
## Key Components
|
| 19 |
|