wolfram
/

Athene-V2-Chat-4.65bpw-h6-exl2

Text Generation

text-generation-inference

Model card Files Files and versions

wolfram commited on Nov 15, 2024

Commit

1502954

·

verified ·

1 Parent(s): 894fb07

Update README.md

Files changed (1) hide show

README.md +3 -0

README.md CHANGED Viewed

@@ -11,6 +11,9 @@ tags:
 base_model:
 - Qwen/Qwen2.5-72B-Instruct
 ---
 # Athene-V2-Chat-72B: Rivaling GPT-4o across Benchmarks
 <p align="center">

 base_model:
 - Qwen/Qwen2.5-72B-Instruct
 ---
+> [!NOTE]
+> EXL2 4.65bpw-h6 quantized version of [Nexusflow/Athene-V2-Chat](https://huggingface.co/Nexusflow/Athene-V2-Chat). Supports 32K context with Q4 cache on systems with 48 GB VRAM.
 # Athene-V2-Chat-72B: Rivaling GPT-4o across Benchmarks
 <p align="center">