Building Local: My 2026 Headless AI Server Journey

Hi everyone!

Just wanted to say it’s great to be here! I’m currently having a blast building and maintaining a local, headless AI server.

There’s something special about running models on your own hardware. I’m curious to know:

  • What are you all currently running on your local setups?

  • Any “hidden gem” projects or tools you think more people should know about in 2026?

Looking forward to catching up with the community!

1 Like

Just use Gemma 4 31B with a smaller Gemma4 for secular decoding

2 Likes