We built a live tracker for the local AI stack (so you don't have to watch 8 GitHubs)

Here is the quiet problem with running AI on your own hardware: the models are the easy part. The hard part is the toolkit around them, and it moves fast.

The project that shrinks a giant model small enough to fit your machine shipped a new quant on Tuesday. The engine that serves it got 30% faster on Wednesday. A new fine-tuning trick landed on Thursday. None of it shows up in the press. It shows up as a GitHub release, seen by the few hundred people watching that repo.

To actually keep up, you would follow about eight projects and read their release notes every morning. Nobody has time for that. So we built the page instead.

What it is

The local AI toolkit is a single page that tracks the open-source projects doing the real work, split into the two jobs that matter:

Fine-tune and train: Unsloth, Axolotl, TorchTune, LLaMA-Factory, TRL. These bend a model to your own data on hardware you own.
Run and serve: llama.cpp, vLLM, SGLang. These load the weights and answer fast, from a single Mac to a production box.

For each one we pull live numbers straight from GitHub: stars, last activity, and the latest tagged release with a plain-English line on what changed. No rumor, no hot takes, no “this changes everything.” Just shipped code, which is the only signal that actually counts. The whole thing refreshes once a day on its own.

Every card ends with a blunt “follow if you…” so you are not left guessing whether a tool is for you.

The part we are most proud of: shop a model by RAM

If you have ever tried to run a local model, you have hit the wall. You find a model you want, you go to download it, and you are staring at a list of cryptic file names: Q4_K_M, UD-IQ2_XXS, Q8_0. Which one fits your machine? The internet’s usual answer is a shrug and some hand-waving about “about half the parameter count.”

We did something simpler and more honest. There is one number that tells you the truth: the size of the quant file. That file has to load into memory, so its size on disk is, in practice, the RAM you need. We read those real sizes directly from Unsloth’s GGUF builds, the open ecosystem’s main quant publisher.

So the page now has a “shop by RAM” table. Pick your hardware from a dropdown, anything from a 24GB GPU to a 512GB Mac Studio, and every quant that does not fit gets dimmed. What is left is exactly what you can run tonight.

A few things you learn fast once you can actually see the numbers:

Q4 is the sweet spot. Q4_K_M gives you most of the quality at about a quarter of the size. It is the default for a reason.
Dynamic quants are magic when you are close. When a model is just over your memory budget, an Unsloth dynamic quant (the ones marked “UD”) squeezes it smaller while keeping quality up. It is how a model that “doesn’t fit” suddenly does.
The biggest models are real, and ruthless. A frontier open model like Kimi K2 runs from about 280GB at its smallest dynamic quant up past 1TB at full size. On a 512GB Mac Studio, only the smaller dynamic quants make it. The table shows you the exact line.

The verdict

If you run models locally, or you are about to, bookmark the toolkit page. It is the page we wished existed: the projects worth following, what they shipped this week, and a straight answer to “will this fit my machine.”

Pair it with our live rankings and you have the whole loop covered. Rankings tell you which model is worth running. The toolkit tells you how to run it and which quant fits. Both refresh daily, both are free, and neither one is trying to sell you anything.

And when you want this kind of setup running privately inside a business, on your own hardware, with someone keeping it healthy, that is exactly what BlueFort IT does for a living.