Hugging Face's logo

adept
/
fuyu-8b

Image-Text-to-Text
Transformers
Safetensors
fuyu
Model card Files
xet
Community
70

Warning output

#15
by dashesy - opened
Discussion
dashesy

It seems harmless but can see this

The attention mask and the pad token id were not set. As a consequence, you may observe unexpected behavior. Please pass your input's attention_mask to obtain reliable results.
Setting pad_token_id to eos_token_id:71013 for open-end generation.

ArthurZ

It’s probably harmless in this case. But we should make sure the attention mask is returned by the Processor @molbap

Molbap

Yes, will be adressed in the follow-up PR on transformers:main :)

Colderthanice

To eliminate attention_mask warning

inputs["attention_mask"] = torch.ones(inputs["input_ids"].shape, device="cuda:0")
generation_output = model.generate(**inputs, max_new_tokens=50, pad_token_id=model.config.eos_token_id)

Molbap

@Colderthanice @dashesy In the recent update with the new transformers release, the attention_mask warning goes away because it is now indeed returned by the Processor :)
Now, batch generation is also supported, so I don't recommend setting attention_mask to torch.ones(inputs["input_ids"].shape)in the current version, that is using left-padding. Let us know if you encounter issues!

Sign up or log in to comment