News:

Willkommen im Notebookcheck.com Forum! Hier können Sie über alle unsere Artikel und allgemein über notebookrelevante Dinge diskutieren. Viel Spass!

Main Menu

Recent posts

#91
News / Re: LG TVs und Monitore sollen...
Last post by tek-check - Today at 11:31:50
As Steve from GN said in his video, this is insane!
A class action law suite against LG should follow as soon as possible in the EU, following a successful court case in Texas.
#92
News / Eine Kamera des Sony-Smartphon...
Last post by Redaktion - Today at 11:30:46
Das neue Flagship-Handy von Sony profitiert von der Expertise der Alpha-Kameras inklusive KI-Assistenten. Nach unserem Testbericht nutzen wir das Android-Handy von Sony für einen Kameravergleich mit dem aktuellen Highend-Phone von Oppo und schauen, ob die Sony-Kameras vielleicht sogar besser sind.

https://www.notebookcheck.com/Eine-Kamera-des-Sony-Smartphones-ist-nicht-konkurrenzfaehig-Test-Xperia-1-VIII-vs-Oppo-Find-X9-Ultra.1330377.0.html
#93
Miscellaneous / Re: Your own ChatGPT, offline:...
Last post by Running AI locally/privat - Today at 11:30:29
Since this may be a big deal, let me mention it here if you don't have enough RAM+VRAM for the mentioned quants:

Announcing Bonsai 27B: The First 27B-Class Model to Run on a Phone
notebookcheck.net/AI-without-the-cloud-This-language-model-now-fits-on-your-iPhone.1345243.0.html
-> prismml.com/news/bonsai-27b

And here's their HF/download page and how to run it under Quickstart:
huggingface.co/prism-ml/Bonsai-27B-gguf
and the local community news about it:
reddit.com/r/LocalLLaMA/comments/1uwfva9/bonsai_27b_1bit_dense_llm_running_locally_in_your/ (to see all comments at once, use old.reddit...)
#94
News / Lenovo releases key specs of L...
Last post by Redaktion - Today at 11:20:04
Lenovo has shared a few key specs of the Legion C700 handheld ahead of its release. The cloud gaming device, which will ship with Tencent's Start cloud gaming service, will have a high refresh rate and drift-free sticks.

https://www.notebookcheck.net/Lenovo-releases-key-specs-of-Legion-C700-cloud-gaming-handheld.1345230.0.html
#95
News / Apple iPhone 18 Pro Max: Log-D...
Last post by Redaktion - Today at 11:18:45
Der umfassende Datenschatz aus dem Tata-Leak konnte nun auch von einem unser neuesten Redakteure analysiert werden. Demnach gibt es kaum noch Zweifel daran, welche Kamera-Upgrades sich im Nachfolger des iPhone 17 Pro Max finden werden. Auch zu den restlichen Kamera-Specs können wir aus erster Hand Auskunft geben.

https://www.notebookcheck.com/Apple-iPhone-18-Pro-Max-Log-Datei-Analyse-bestaetigt-zwei-grosse-Kamera-Upgrades.1345219.0.html
#96
News / LG TVs and monitors said to su...
Last post by Redaktion - Today at 11:18:10
Anyone who owns an LG smart TV must inform all guests and family members that they are being monitored – this is required by LG's current terms of use. Meanwhile, LG monitors install potential malware and surveillance software on a connected Windows computer.

https://www.notebookcheck.net/LG-TVs-and-monitors-said-to-surveil-users-and-install-bloatware-without-asking.1345261.0.html
#97
News / Re: AI without the cloud: This...
Last post by correction - Today at 11:11:53
Quotet's based on Alibaba's open-source Qwen3.6 27B
It's open-weight, not open-source. Similar to what would be a freeware .exe file. There are also open-source AI models.
#98
News / LG TVs und Monitore sollen Nut...
Last post by Redaktion - Today at 11:11:14
Wer einen LG Smart TV besitzt, muss alle Gäste und Familienmitglieder darüber informieren, dass diese abgehört werden – das verlangen LGs aktuelle Nutzungsbedingungen. Monitore installieren unterdessen potenzielle Malware und Überwachungssoftware auf einem angeschlossenen Windows-Computer.

https://www.notebookcheck.com/LG-TVs-und-Monitore-sollen-Nutzer-ueberwachen-ungefragt-Bloatware-installieren.1345252.0.html
#99
News / Casio launcht drei neue kompak...
Last post by Redaktion - Today at 11:05:19
Casio hat in den USA die komplette Baby-G BG169CM-Camouflage-Kollektion vorgestellt. Sie umfasst drei Modelle: die BG169CM-2 in Türkis (145 US-Dollar/ 119 Euro), die BG169CM-4 in Pink (145 US-Dollar/ 119 Euro) und die BG169CMB-8 in Grau-Camouflage (155 US-Dollar/ 129 Euro). Alle drei Uhren sind bis 200 Meter wasserdicht und bieten eine Weltzeitfunktion mit Unterstützung für 29 Zeitzonen.

https://www.notebookcheck.com/Casio-launcht-drei-neue-kompakte-Camouflage-Uhren-fuer-den-US-Markt.1345251.0.html
#100
Miscellaneous / Re: Your own ChatGPT, offline:...
Last post by Running AI locally/privat - Today at 11:00:37
QuoteStart with LM Studio and Qwen3 8B. It runs on most reasonably current laptops and is more than enough for everyday use.
Has the whole post been written by (outdated) AI or why would you recommend this outdated small AI model? It may leave people disappointed in local AI.

Alright, here we go:
Running AI locally and hence privately requires these 2-3 things:
1. Memory size to fit a decently capable LLM model + its context. Fitting a AI model in the first place is the most important metric or you can not run it at all.
The mentioned Qwen3-8B and even Qwen3-14B are a questionable suggestion, as they may will leave a disappointing impression about local AI. I recommend you see if you can fit a quant of the newer Qwen3.6-27B or Qwen3.6-35B, see below. Yes, the same quant of the 27B vs 14B LLM model will require almost 2 times the memory.
2. Memory speed, also known as memory bandwidth: Relevant for token generation (output) speed.
3. GPU 3D/FPS performance / compute: Relevant for prompt processing (input) speed. But GPU performance is determined by the memory bandwidth. Given the same VRAM amount, a faster GPU in terms of FPS will have a faster prompt processing (note, there are a few cases of 5070 12 GB VRAM vs 5060 Ti 16 GB VRAM).

Current SOTA AI LLM models, for their size, are:
huggingface.co/Qwen/Qwen3.6-27B (dense architecture) (Downloads last month: 5.0 million)
and
huggingface.co/Qwen/Qwen3.6-35B-A3B (MoE architecture) (Downloads last month: 5.6 million).

Dense vs MoE
In a dense 27B model, all 27B parameters per token are activated vs in the 35B-A3B, where only 3B active are used. Hence, for the same size, a dense model performs much better[2], but the 35B-A3B will be running faster.

Popular, bang for the buck, quants are (a UD-Q4_K_XL quant is good for 80k-120k of context):
huggingface.co/unsloth/Qwen3.6-35B-A3B-MTP-GGUF UD-Q4_K_XL (22.9 GB)
and
huggingface.co/unsloth/Qwen3.6-27B-MTP-GGUF UD-Q4_K_XL (17.9 GB).

Required memory to run these quants: Quant size + 2 GB for 32,000 and 8 GB for 128,000 tokens of context (linear)[1].

If you have 32 GB RAM and no 8 GB VRAM GPU: 32 GB RAM - 8 GB for the OS itself = 24 GB free for fitting an AI LLM model.
MoE 35-A3B model:
24 GB - 22.9 GB = ~ 1 GB for the context. Based on [1], this gives you about 16,000 tokens of context. It is likely that you can squeeze out another GB or more if the OS is fine with it, so it would be 32,000 tokens of context, or slightly more. Another confirmation:
Quote from: reddit.com/r/LocalLLaMA/comments/1sq94qx/is_anyone_getting_real_coding_work_done_with.. I've come to the conclusion that (1) 32768 is the biggest context I can get away with in an adequately smart model, and (2) it just ain't enough.
Dense 27B model:
The dense 27B model quant will run slower, but leave you with about 6 GB for the context, which should allow for over 90,000 tokens of context.

Large text inputs of 100,000 tokens (1 token = 0.75 words) or agentic workflows, which also may require over 100,000 tokens, will need additional 8 GB for 128,000 context tokens[1]. This is where your 32 GB RAM will need additional 8 GB of memory. In a no-GPU laptop, that would require a RAM upgrade to 2 x 24 GB RAM. In a PC, you'd be better off getting a 8 GB VRAM, or more, GPU. If you don't have a laptop yet, but plan one for AI, get one with a 8 GB VRAM GPU (a laptop with more than 8 GB VRAM GPU starts to become quite expensive, to the point where it's much cheaper to get a desktop PC) and put 2 x 16 GB RAM in it (= 32 GB RAM + 8 GB VRAM). A GPU will also have the benefit of giving you faster prompt processing and token generation.

To run these models you need a so called inference engine. The most popular is llama.cpp. The mentioned LM Studio and Ollama are just wrappers for llama.cpp and may lag behind in feature support. Another wrapper to recommend that is more private and does not call home at startup, is TextGen. I started with TextGen (it had a different name back then), now I'm using llama.cpp's nice webUI (which is actively worked on: github.com/ggml-org/llama.cpp/commits/master/tools/ui).

More, up to 300B parameters, open-weight models evaluated:
artificialanalysis.ai/?models=qwen3-5-122b-a10b-non-reasoning%2Cqwen3-5-122b-a10b%2Cmimo-v2-5-0424%2Cqwen3-6-27b-non-reasoning%2Cminimax-m2-7%2Cqwen3-6-35b-a3b-non-reasoning%2Cstep-3-7-flash%2Cqwen3-6-35b-a3b%2Cdeepseek-v4-flash-high%2Cqwen3-6-27b%2Cdeepseek-v4-flash-non-reasoning%2Chy3%2Cgemma-4-31b%2Cgemma-4-31b-non-reasoning%2Cdeepseek-v4-flash%2Cmistral-medium-3-5%2Cgpt-oss-120b%2Cnvidia-nemotron-3-super-120b-a12b&intelligence=artificial-analysis-intelligence-index

News related to running open-weight AI LLM models locally and privately: reddit.com/r/LocalLLaMA/top, reddit.com/r/LocalLLM/top.

[1] reddit.com/r/LocalLLaMA/comments/1tvluaj/how_much_vram_needed_for_qwen_36_27b_q8_with_262k
[2] See 27B dense vs 122B-A10B (MoE) evaluation: huggingface.co/Qwen/Qwen3.5-122B-A10B#benchmark-results. The 122B required 4.5 times more memory, but they score about the same.