Models for local deployment - a yjernite Collection

yjernite 's Collections

Text embedding models I use

super-smol to fine-tune

Models for local deployment

Inference-supported production models

Document processing

Privacy

Models for local deployment

updated Sep 2

Qwen/Qwen3-4B-Instruct-2507

Text Generation • 4B • Updated Sep 17 • 3.45M • • 417
unsloth/Qwen3-4B-Instruct-2507-GGUF

4B • Updated Aug 20 • 52.7k • 84
HuggingFaceTB/SmolLM3-3B

Text Generation • 3B • Updated Sep 10 • 61.3k • • 745
unsloth/SmolLM3-3B-GGUF

3B • Updated Jul 8 • 919 • 15
Qwen/Qwen3-Embedding-4B-GGUF

4B • Updated Aug 3 • 8.04k • 65

Note ./build/bin/llama-server -hf Qwen/Qwen3-Embedding-4B-GGUF:Q6_K --port 8080 --pooling last --verbose-prompt --parallel 4 --batch-size 8 --embedding --ctx-size 8000 --cache-type-k q4_0 --gpu-layers 36
Qwen/Qwen3-Embedding-0.6B

Feature Extraction • 0.6B • Updated Jun 20 • 3.68M • • 679
google/gemma-3-270m

Text Generation • 0.3B • Updated Aug 14 • 124k • 886