2k bucks for 16 GB soldered..yeah..at least it hasn't an "AI" in its product name, because to run multi purpose, generally capable, AI LLM models, and depending on context, at the very least 32 GB RAM are necessary, but better 32 GB RAM + 6-8 GB VRAM GPU: Which leads us to a generic gaming laptop, that costs 700-800 used and new still much cheaper, being much superior for running AI LLM models locally VS this one. A 6-8 GB VRAM gaming laptop will not only provide the important 6-8 (fast) additional memory, but also automatically have more compute, which will lead to faster prompt processing (input) and offloaded to 6-8 GB VRAM also to faster token generation (output).
Quote from: reddit.com/r/LocalLLaMA/comments/1sq94qx/is_anyone_getting_real_coding_work_done_with.. I've come to the conclusion that (1) 32768 is the biggest context I can get away with in an adequately smart model, and (2) it just ain't enough.
More about running AI models locally: notebookchat.com/index.php?topic=315954.0 ("Your own ChatGPT, offline: AI without the cloud on your laptop") and comments.
English
Español
Français
Italiano
Nederlands
Polski
Português
Русский
Türkçe
Svenska
Chinese
Magyar