YAML Metadata Warning: empty or missing yaml metadata in repo card (https://huggingface.co/docs/hub/model-cards#model-card-metadata)

Instella 3B FP16

This is a FP16 version of amd/Instella-3B-Instruct.

Quantization: FP16 (half precision) Use case: CPU or GPU inference with reduced memory usage.

Example Usage

from transformers import AutoTokenizer, AutoModelForCausalLM
tokenizer = AutoTokenizer.from_pretrained("username/instella-3b-fp16")
model = AutoModelForCausalLM.from_pretrained("username/instella-3b-fp16", torch_dtype="float16")

prompt = "Explain quantum computing in simple terms."
inputs = tokenizer(prompt, return_tensors="pt")
print(tokenizer.decode(model.generate(**inputs, max_new_tokens=100)[0]))

Downloads last month: 3

Safetensors

Model size

3B params

Tensor type

F16

Inference Providers NEW

This model isn't deployed by any Inference Provider. 🙋 Ask for provider support