Quote from: opckieran on Yesterday at 23:54:26$2100 is WILD for 1200p and only 32GB of RAM.
Wild indeed.
32 GB RAM - 8 GB for the OS itself = 24 GB free for fitting an AI LLM model.
Current SOTA AI LLM models, for their size, are:
huggingface.co/Qwen/Qwen3.6-27B (dense architecture) (Downloads last month: 5.0 million)
and
huggingface.co/Qwen/Qwen3.6-35B-A3B (MoE architecture) (Downloads last month: 5.6 million).
Popular, bang for the buck, quants are:
huggingface.co/unsloth/Qwen3.6-27B-MTP-GGUF UD-Q4_K_XL (17.9 GB)
and
huggingface.co/unsloth/Qwen3.6-35B-A3B-MTP-GGUF UD-Q4_K_XL (22.9 GB) (good for 80k-120k of context).
Dense vs MoE
In a dense 27B model, all 27B parameters per token are activated vs in the 35B-A3B, where only 3B active are used. Hence, for the same size, a dense model performs much better[2], but the 35B-A3B will be running faster.
Let's say, because this laptop has no fast VRAM, one wants to use the 35B-A3B model, because it will run faster than the dense 27B model: 22.9 GB barely fits into the left 24 GB RAM. Theoretically, 1 GB allows for about 16,000 of context[1].
And it and the 32 GB RAM confirms this quote:
Quote from: reddit.com/r/LocalLLaMA/comments/1sq94qx/is_anyone_getting_real_coding_work_done_with.. I've come to the conclusion that (1) 32768 is the biggest context I can get away with in an adequately smart model, and (2) it just ain't enough.
Agentic workflows may require over 100,000 of context. So fix this, laptops really needs to have at least 48 GB RAM.
The 27B quant will run slower, but leave one with about 6 GB for the context, which should allow for over 90,000 tokens of context.
[1] reddit.com/r/LocalLLaMA/comments/1tvluaj/how_much_vram_needed_for_qwen_36_27b_q8_with_262k
[2] See 27B dense vs 122B-A10B (MoE) evaluation: huggingface.co/Qwen/Qwen3.5-122B-A10B#benchmark-results. The 122B required 4.5 times more memory, but they score about the same.
Deutsch
English
Español
Français
Italiano
Nederlands
Polski
Português
Русский
Türkçe
Svenska
Chinese
Magyar