Update README.md

README.md

@@ -1,3 +1,68 @@
----
-license: apache-2.0
----
+---
+license: apache-2.0
+pipeline_tag: text-generation
+tags:
+ - ONNX
+ - DML
+ - ONNXRuntime
+ - mistral
+ - conversational
+ - custom_code
+inference: false
+---
+
+# Mistral-7B-Instruct-v0.2 ONNX models
+
+<!-- Provide a quick summary of what the model is/does. -->
+This repository hosts the optimized versions of [Mistral-7B-Instruct-v0.2](https://huggingface.co/mistralai/Mistral-7B-Instruct-v0.2) to accelerate inference with ONNX Runtime.
+
+The Mistral-7B-Instruct-v0.2 Large Language Model (LLM) is an instruct fine-tuned version of the Mistral-7B-v0.2.
+
+Optimized Mistral models are published here in [ONNX](https://onnx.ai) format to run with [ONNX Runtime](https://onnxruntime.ai/) on CPU and GPU across devices, including server platforms and Windows, Linux, and Mac desktops, with the precision best suited to each of these targets.
+
+[DirectML](https://aka.ms/directml) support lets developers bring hardware acceleration to Windows devices at scale across AMD, Intel, and NVIDIA GPUs. Along with DirectML, ONNX Runtime provides cross platform support for Mistral across a range of devices for CPU and GPU.
+
+To easily get started with Mistral, you can use [Olive](https://github.com/microsoft/Olive), our easy-to-use, hardware-aware model optimization tool. See [here](https://github.com/microsoft/Olive/tree/main/examples/mistral) for instructions on how to run it with Mistral.
+
+## ONNX Models 
+
+Here are some of the optimized configurations we have added:  
+
+1. ONNX model for int4 DML: ONNX model for AMD, Intel, and NVIDIA GPUs on Windows, quantized to int4 using [AWQ](https://arxiv.org/abs/2306.00978).
+2. ONNX model for fp16 CUDA: ONNX model you can use to run for your NVIDIA GPUs.
+3. ONNX model for int4 CUDA: ONNX model for NVIDIA GPUs using int4 quantization via RTN.
+4. ONNX model for int4 CPU: ONNX model for your CPU, using int4 quantization via RTN.
+
+## Hardware Supported
+
+The models are tested on:
+- GPU SKU: RTX 4090 (DirectML)
+- GPU SKU: 1 A100 80GB GPU, SKU: Standard_ND96amsr_A100_v4 (CUDA)
+- CPU SKU: Standard F64s v2 (64 vcpus, 128 GiB memory)
+
+Minimum Configuration Required:
+- Windows: DirectX 12-capable GPU and a minimum of 4GB of combined RAM
+- CUDA: Streaming Multiprocessors (SMs) >= 70 (i.e. V100 or newer)
+
+### Model Description
+
+- **Developed by:**  Microsoft
+- **Model type:** ONNX
+- **Language(s) (NLP):** Python, C, C++
+- **License:** Apache License Version 2.0
+- **Model Description:** This is a conversion of the Mistral-7B-Instruct-v0.2 model for ONNX Runtime inference.
+
+## Additional Details
+- [**Mistral Model Announcement Link**](https://mistral.ai/news/announcing-mistral-7b/)
+- [**Mistral Model Card**](https://huggingface.co/mistralai/Mistral-7B-Instruct-v0.2)
+- [**Mistral Technical Report**](https://arxiv.org/abs/2310.06825)
+
+## Appendix
+
+### Activation Aware Quantization
+
+AWQ works by identifying the top 1% most salient weights that are most important for maintaining accuracy and quantizing the remaining 99% of weights. This leads to less accuracy loss from quantization compared to many other quantization techniques. For more on AWQ, see [here](https://arxiv.org/abs/2306.00978).
+
+
+## Model Card Contact
+sschoenmeyer, sunghcho, kvaishnavi