AIDC-AI
/

Ovis1.6-Llama3.2-3B-GPTQ-Int4

Image-Text-to-Text

text-generation

4-bit precision

Model card Files Files and versions

TryingHard commited on Nov 8, 2024

Commit

45291da

·

verified ·

1 Parent(s): 7d563ef

Update README.md

Files changed (1) hide show

README.md +1 -1

README.md CHANGED Viewed

@@ -66,7 +66,7 @@ from auto_gptq.modeling import OvisLlamaGPTQForCausalLM
 # load model
 load_device = "cuda:0" # customize load device
-model = OvisLlamaGPTQForCausalLM.from_pretrained(
     "AIDC-AI/Ovis1.6-Llama3.2-3B-GPTQ-Int4",
     device=load_device,
     trust_remote_code=True

 # load model
 load_device = "cuda:0" # customize load device
+model = OvisLlamaGPTQForCausalLM.from_quantized(
     "AIDC-AI/Ovis1.6-Llama3.2-3B-GPTQ-Int4",
     device=load_device,
     trust_remote_code=True