The 8 GB unified memory/RAM don't allow it to fit/run the minimally small, yet good Q4_K_M quant (e.g. huggingface.co/unsloth/Qwen3.8-27B-GGUF/blob/main/Qwen3.8-27B-UD-Q4_K_M.gguf (to run it use e.g. llama.cpp (it has a nice webUI) or e.g. Unsloth Desktop, that also uses llama.cpp as its backend and both are open source)) of smart, general purpose, 24B to 35B (e.g. Qwen3.6-35B-A3B (recommended minimum quant is Q4_K_M)) AI models, see e.g.:
artificialanalysis.ai/models/open-source/small (maybe LFM2.5-24B-A2B will also be evaluated there soon)
And these ~30B parameters models are no joke:
Quote from: reddit.com/r/LocalLLaMA/comments/1x4cm73/once_again_anthropic_doing_free_marketing_for/Once again anthropic doing free marketing for open weight models
Models of 30B parameters and under can already complete many of the individual steps for replication today, though they make mistakes when composing them end to end. In our simplified evaluations, GLM-5.2 and stronger models are capable of end to end replication. If small models improve at their historical rate, they'll match this capability level by the end of this year. Small weight-based replicators may be possible soon after.
English
Español
Français
Italiano
Nederlands
Polski
Português
Русский
Türkçe
Svenska
Chinese
Magyar