ig1sa commited on
Commit
122c7c0
·
verified ·
1 Parent(s): ed56e8c

Update README.md

Browse files
Files changed (1) hide show
  1. README.md +12 -1
README.md CHANGED
@@ -10,4 +10,15 @@ base_model:
10
  - Qwen/Qwen2.5-VL-7B-Instruct
11
  ---
12
 
13
- [FP8 activation quantization](https://github.com/vllm-project/llm-compressor/tree/main/examples/quantization_w8a8_fp8) performed with [llm-compressor](https://github.com/vllm-project/llm-compressor)
 
 
 
 
 
 
 
 
 
 
 
 
10
  - Qwen/Qwen2.5-VL-7B-Instruct
11
  ---
12
 
13
+ ```bash
14
+ # Create a dedicated python env
15
+ python3 -m venv llmcompressor
16
+ source llmcompressor/bin/activate
17
+ # Install llm-compressor and additionnal needed libs
18
+ pip install llmcompressor qwen_vl_utils torchvision
19
+ # Download model in HF cache
20
+ hf download Qwen/Qwen2.5-VL-7B-Instruct
21
+ # Start quantization
22
+ wget https://github.com/vllm-project/llm-compressor/blob/main/examples/quantization_w8a8_fp8/qwen_2_5_vl_example.py -O qwen_2_5_vl_fp8.py
23
+ python3 qwen_2_5_vl_fp8.py
24
+ ```