GGUF for nvidia cards?
#20
by krigeta - opened
anyone?
Yes, there are two sets of models quantized in GGUF. Rodrigomt's (https://huggingface.co/rodrigomt/s2-pro-gguf) and mine (https://huggingface.co/mach9243/s2-pro-gguf). They differ in their uses and objectives, but both support Nvidia GPUs (mine use Vulkan). See "Quantizations"