TheStageAI
/

Elastic-Llama-3.2-1B-Instruct

Text Generation

text2text-generation

Model card Files Files and versions

psynote123 commited on May 22

Commit

f83ab7d

·

verified ·

1 Parent(s): 8d4521e

Update README.md

Files changed (1) hide show

README.md +1 -1

README.md CHANGED Viewed

@@ -97,7 +97,7 @@ chat_prompt = tokenizer.apply_chat_template(
 inputs = tokenizer(chat_prompt, return_tensors="pt")
 inputs.to(device)
-with torch.inference_mode:
     generate_ids = model.generate(**inputs, max_length=500)
 input_len = inputs['input_ids'].shape[1]

 inputs = tokenizer(chat_prompt, return_tensors="pt")
 inputs.to(device)
+with torch.inference_mode():
     generate_ids = model.generate(**inputs, max_length=500)
 input_len = inputs['input_ids'].shape[1]