ISTA-DASLab

/

Llama-3.1-8B-Instruct-MR-GPTQ-nvfp

like 0

Follow

IST Austria Distributed Algorithms and Systems Lab 117

This PR significantly enhances the model card for Llama-3.1-8B-Instruct-MR-GPTQ-nvfp by:

• Correcting Metadata: The pipeline_tag has been updated from image-text-to-text to text-generation, accurately reflecting the model's functionality as an LLM for language tasks. The license remains llama3.1, consistent with the base model.
• Adding Essential Links: Direct links to the paper (Bridging the Gap Between Promise and Performance for Microscaling FP4 Quantization) and the associated GitHub repository (https://github.com/IST-DASLab/FP-Quant) have been added prominently at the top of the model card for improved discoverability.
• Including Sample Usage: Concrete code snippets for inference using both transformers and vLLM have been integrated into the "Usage" section. These examples, sourced directly from the project's GitHub README, provide immediate utility for users.
• Adding Citation: A BibTeX citation has been included at the end of the model card, ensuring proper academic attribution for the work.

These changes provide a more accurate, comprehensive, and user-friendly model card. Please review and merge if everything looks good.