Updated Dynamic DeepSeek-V3.1 GGUFs - even better performance than before! 🐋
Hey guys, we reuploaded the DeepSeek-V3.1 quants and according to 3rd party Aider polyglot benchmarks, they're even better than before.
The benchmarks are 90% done already and we compared them other quants and our previous quants and the results are clearly an improvement. Thanks to neonilithic for doing the benchmarks!
We converted DeepSeek-V3.1 using our normal conversion, however we needed to update it as we didn't know llama.cpp overrode some of our layer quantization for conversion so we needed to change reupload them. The quants should only be a few MB bigger but the increase in accuracy is very large.
We'll announce the amazing results likely next week, yes you will need to redownload.
That'a boy Mikey! 😋👏 🦥
This new version not upload yet? or the current version is the version?
The current version is the new version!
Hey guys, incase you were interested, we released benchmarks for DeepSeek-V3.1 GGUF on Aider Polyglot: https://x.com/UnslothAI/status/1965797776387879378
Blog with details: https://docs.unsloth.ai/basics/unsloth-dynamic-ggufs-on-aider-polyglot
CC: @qingy2024 @mrtime-ai @Joseph717171 @CalvinZero @ronedgecomb
Thanks for share this great report.
Is there a plan to add kimi-k2-0905 into the report ?
Thanks for share this great report.
Is there a plan to add kimi-k2-0905 into the report ?
Unfortunately that's unlikely as the model is sooooo large but expect to see similar accuracy recovery