David Soušek
sousekd
AI & ML interests
Running AI on-prem with agents processing confidential data. Supporting government, private sector, and all kinds of weirdos.
Recent Activity
new activity
about 2 hours ago
ubergarm/Devstral-2-123B-Instruct-2512-GGUF:Going to fit it into RTX PRO 6000!
liked
a model
2 days ago
AesSedai/GLM-4.6-Derestricted-GGUF
replied to
csabakecskemeti's
post
6 days ago
Looking for some help to test an INT8 Deepseek 3.2:
SGLang supports Channel wise INT8 quants on CPUs with AMX instructions (Xeon 5 and above AFAIK)
https://lmsys.org/blog/2025-07-14-intel-xeon-optimization/
Currently uploading an INT8 version of Deepseek 3.2 Speciale:
https://huggingface.co/DevQuasar/deepseek-ai.DeepSeek-V3.2-Speciale-Channel-INT8
I cannot test this I'm on AMD
"AssertionError: W8A8Int8LinearMethod on CPU requires that CPU has AMX support"
(I assumed it can fall back to some non optimized kernel but seems not)
If anyone with the required resources (Intel Xeon 5/6 + ~768-1TB ram) can help to test this that would be awesome.
If you have hints how to make this work on AMD Threadripper 7000 Pro series please guide me.
Thanks all!
Organizations
None yet