casinca commited on
Commit
ddafa85
·
verified ·
1 Parent(s): 47dfcb6

docs: Updated the `Transformers` example to use intended temp=0.15

Browse files

This PR adds the required hparam arguments to enable stochastic sampling (temp 0.15) for the `transformers` snippet rather than greedy decoding.
This reflects the recommended usage pattern intended by Mistral.

fix: https://huggingface.co/mistralai/Devstral-2-123B-Instruct-2512/discussions/9

Files changed (1) hide show
  1. README.md +2 -0
README.md CHANGED
@@ -487,6 +487,8 @@ input_ids = tokenized["input_ids"].to(device="cuda")
487
  output = model.generate(
488
  input_ids,
489
  max_new_tokens=200,
 
 
490
  )[0]
491
 
492
  decoded_output = tokenizer.decode(output[len(tokenized["input_ids"][0]) :])
 
487
  output = model.generate(
488
  input_ids,
489
  max_new_tokens=200,
490
+ do_sample=True,
491
+ temperature=0.15,
492
  )[0]
493
 
494
  decoded_output = tokenizer.decode(output[len(tokenized["input_ids"][0]) :])