Update README.md

README.md

@@ -1,13 +1,87 @@
-# Exl2 quants for [Llama-Guard-3-1B](https://huggingface.co/meta-llama/Llama-Guard-3-1B)
-
-## Automatically quantized using the auto quant script from [hf-scripts](https://huggingface.co/anthonyg5005/hf-scripts)
-
-### BPW:
-
-[4.0](https://huggingface.co/Anthonyg5005/Llama-Guard-3-1B-exl2/tree/4.0bpw)\
-[4.5](https://huggingface.co/Anthonyg5005/Llama-Guard-3-1B-exl2/tree/4.5bpw)\
-[5.0](https://huggingface.co/Anthonyg5005/Llama-Guard-3-1B-exl2/tree/5.0bpw)\
-[6.0](https://huggingface.co/Anthonyg5005/Llama-Guard-3-1B-exl2/tree/6.0bpw)\
-[6.5](https://huggingface.co/Anthonyg5005/Llama-Guard-3-1B-exl2/tree/6.5bpw)\
-[8.0](https://huggingface.co/Anthonyg5005/Llama-Guard-3-1B-exl2/tree/8.0bpw)\
-[measurement.json](https://huggingface.co/Anthonyg5005/Llama-Guard-3-1B-exl2/blob/main/measurement.json)
+---
+language:
+- en
+- de
+- fr
+- it
+- pt
+- hi
+- es
+- th
+library_name: exllamav2
+pipeline_tag: text-generation
+tags:
+- facebook
+- meta
+- pytorch
+- llama
+- llama-3
+- safety
+license: llama3.2
+base_model:
+- meta-llama/Llama-Guard-3-1B
+---
+# Exl2 quants for [Llama-Guard-3-1B](https://huggingface.co/meta-llama/Llama-Guard-3-1B)
+
+This is a quantized model from the llama guard series for use with [exllamav2](https://github.com/turboderp/exllamav2).\
+Please refer to the original model card for instructions on how to use this model.
+
+### BPW:
+
+[4.0](https://huggingface.co/Anthonyg5005/Llama-Guard-3-1B-exl2/tree/4.0bpw)\
+[4.5](https://huggingface.co/Anthonyg5005/Llama-Guard-3-1B-exl2/tree/4.5bpw)\
+[5.0](https://huggingface.co/Anthonyg5005/Llama-Guard-3-1B-exl2/tree/5.0bpw)\
+[6.0](https://huggingface.co/Anthonyg5005/Llama-Guard-3-1B-exl2/tree/6.0bpw)\
+[6.5](https://huggingface.co/Anthonyg5005/Llama-Guard-3-1B-exl2/tree/6.5bpw)\
+[8.0](https://huggingface.co/Anthonyg5005/Llama-Guard-3-1B-exl2/tree/8.0bpw)\
+[measurement.json](https://huggingface.co/Anthonyg5005/Llama-Guard-3-1B-exl2/blob/main/measurement.json)
+
+# How to download:
+
+### oobabooga's downloader
+
+use something like [download-model.py](https://github.com/oobabooga/text-generation-webui/blob/main/download-model.py) to download with python requests.\
+Install requirements:
+
+```shell
+pip install requests tqdm
+```
+
+Example for downloading 8bpw:
+
+```shell
+python download-model.py Anthonyg5005/Llama-Guard-3-1B-exl2:8.0bpw
+```
+
+### huggingface-cli
+
+You may also use huggingface-cli\
+To install it, install python hf-hub
+
+```shell
+pip install huggingface-hub
+```
+
+Example for 8bpw:
+
+```shell
+huggingface-cli download Anthonyg5005/Llama-Guard-3-1B-exl2 --local-dir Llama-Guard-3-1B-exl2-exl2-8bpw --revision 8.0bpw
+```
+### Git LFS (not recommended)
+
+I would recommend the http downloaders over using git, they can resume downloads if failed and are much easier to work with.\
+Make sure to have git and git LFS installed.\
+Example for 8bpw download with git:
+
+Have LFS file skip disabled
+```shell
+# windows
+set GIT_LFS_SKIP_SMUDGE=0
+# linux
+export GIT_LFS_SKIP_SMUDGE=0
+```
+
+Clone repo branch
+```shell
+git clone https://huggingface.co/Anthonyg5005/Llama-Guard-3-1B-exl2 -b 8.0bpw
+```