Why this is good news and why 192, or better, 256 GB RAM is needed over 128 GB RAM:
For the same file-size and hence memory requirement, a dense AI LLM model outperforms a MoE AI LLM model by a lot, so a equally performing MoE model is multiple times larger, see 122B MoE vs 27B dense evaluation comparison: huggingface.co/Qwen/Qwen3.5-122B-A10B#benchmark-results, a difference of 122B/27B = 4.5 times in memory requirement. The advantage is that this 122B(-A10B) MoE model runs faster doe to fewer active parameters (27B vs 10B). And even larger ones, like 300B to maybe 450B, MoE models, run faster if they have fewer active parameters (e.g.: MiniMax-M3 (426B-23B)).
So, to significantly outperform a 27B dense in all metrics, a larger than 120B MoE model is needed.
Fortunately, there are many ~300 B MoE models to choose from. Here is a list of evaluated by AA (also see how good the much smaller 27B dense performs):
artificialanalysis.ai/?models=qwen3-5-122b-a10b-non-reasoning%2Cqwen3-5-122b-a10b%2Cmimo-v2-5-0424%2Cqwen3-6-27b-non-reasoning%2Cminimax-m2-7%2Cnvidia-nemotron-3-super-120b-a12b%2Cqwen3-6-35b-a3b-non-reasoning%2Cstep-3-7-flash%2Cqwen3-6-35b-a3b%2Cdeepseek-v4-flash-high%2Cqwen3-6-27b%2Cdeepseek-v4-flash-non-reasoning%2Chy3%2Cgemma-4-31b%2Cgemma-4-31b-non-reasoning%2Cdeepseek-v4-flash%2Cmistral-medium-3-5%2Cgpt-oss-120b%2Cmotif-0714&intelligence=artificial-analysis-intelligence-index
Even a 120B model, at an 8-bit (quant), requires 120 GB of memory + context, so to fit these ~300B to ~400B models into 192 GB RAM, 2- to 5-bit quants are used.
Running ~400B MoE models at a 3-bit quant is also a possibility: E.g. MiniMax-M3 426B IQ3_XXS quant (159 GB).
Additionally, dense models are rare and the direction everyone seems to have chosen is MoE models.
Not evaluated yet by AA, because just released, but maybe later:
Solar-Open2 250B-A15B MoE
Laguna-S-2.1 - 120B-A8B MoE
More news about these models at reddit.com/r/LocalLLaMA.
AMD knows that more than 120B parameter models are needed and already exist and that their 2- to 5-bit quants perform well and this is why they announced Gorgon Halo (a Strix Halo rename/refresh, but with up to 192 GB RAM (160 GB allocatable on Windows, and maybe more on Linux, just like with Strix Halo)): notebookcheck.net/AMD-Ryzen-AI-Max-400-lineup-now-official-with-up-to-192-GB-RAM.1301777.0.html, which sees its (one of the) first implementation in this very news by FRAMEWORK.
A bit of recent history: For the same evaluation performance, MoE models' fewer active parameters were supposed to fix dense model's speed. RAM is typically much slower than VRAM, but also much larger than VRAM, so it was a perfect match. This is why MoE models came into being. But since recently, RAM prices have increased like 4 times and the advantage kinda went away (from this very news: ".. ends up costing over $4,000"). We need more dense models again, because memory prices are high.
PS: Most, if not all, desktop PC, mid-range B850 motherboards, support up to 256 GB RAM, but their memory bandwidth is less than half of this Halo product. Still, you can and should then also add 1 or even more (high VRAM) GPUs. This will compensate for the slower RAM by adding much faster VRAM and you end up having more total memory, too.
Deutsch
English
Español
Français
Italiano
Nederlands
Polski
Português
Русский
Türkçe
Svenska
Chinese
Magyar