NotebookCHECK - Notebook Forum

English => Reviews => Topic started by: Redaktion on September 18, 2025, 04:25:15

Title: Framework Desktop review: Mini PC wrapped in a mini-ITX body
Post by: Redaktion on September 18, 2025, 04:25:15
Framework is betting big on AMD yet again and the result is a PC with nearly all the repairability benefits of a standard Mini-ITX system while being almost as small as a mini PC.

https://www.notebookcheck.net/Framework-Desktop-review-Mini-PC-wrapped-in-a-mini-ITX-body.1115803.0.html
Title: Re: Framework Desktop review: Mini PC wrapped in a mini-ITX body
Post by: Running AI LLMs locally on September 18, 2025, 10:00:21
QuoteFramework Desktop Ryzen AI Max
AMD Ryzen AI Max+ 395, Radeon 8060S   
123082 MB/s

Geekom A9 Max, AI 9 HX 370
AMD Ryzen AI 9 HX 370, Radeon 890M   
86541 MB/s
HX 370: 128-bit * 5600 MT/s / 1000 / 8 = 89.6 GB/s (does check out with the measurement)
AI Max+ 395: 256-bit * 8000 MT/s / 1000 / 8 = 256 GB/s (does NOT check out with the measurement)
Can you measure with the newest version 8.0?

QuoteUnfortunately, the system is not compatible with the recent AMD FSR4 update as the GPU is based on the same RDNA 3.5 architecture as the Radeon 890M and not the Radeon RX 9000 RDNA4 series.
Imagine paying 1600 bucks for last gen GPU architecture.

LLMs
The "Strix Halo" APU is a 256-bit chip with a theoretical memory bandwidth of 256 GB/s (256-bit * 8000 MT/s / 1000 / 8) (and ~210 GB/s practically (expected)), comparable to an entry level quad-channel (4 * 64-bit) workstation' memory bandwidth. A normal desktop PC is dual-channel at best. AMD specifically advertises "Strix Halo" for running/inferencing LLMs. You can run the same LLMs on any PC, if you have at least the same amount of RAM (well, running off of a SSD will also work, but the speed will be super slow), ATX sized or not, dual-channel RAM or not, the differences are:
   
Using the relatively expensive Strix Halo APU/chip, but giving it only 64 GB RAM is wasting expensive silicon, because it's simply not enough for many LLMs (btw: the memory bandwidth will be the same, they are just using less dense RAM chips): Give it at least 96 GB RAM or just the full 128 GB, because on Windows, only 75 % can be allocated to a LLM (as far as I know, maybe it has changed) (that's 96 GB out of 128 GB RAM total).

Questions to ask yourself:
Title: Re: Framework Desktop review: Mini PC wrapped in a mini-ITX body
Post by: heffeque on September 18, 2025, 17:55:17
Love how silent it is (especially with the Noctua printed mod), yet powerful enough for stuff that my previous PC couldn't handle.

It's perfect for my HTPC setup. Really happy camper.
Title: Re: Framework Desktop review: Mini PC wrapped in a mini-ITX body
Post by: Julian M on September 18, 2025, 21:05:31
So odd to have a box this relatively big for nothing but a hefty cooler and PSU - there's no room for extra storage or cards, you get two m.2 slots and that's it.

Definitely not a great approach, despite Apple doing this all-soldered thing for years now.
Title: Re: Framework Desktop review: Mini PC wrapped in a mini-ITX body
Post by: Running AI LLMs locally on September 19, 2025, 09:37:33
Julian M, any Strix Halo implementations that do have the extra storage? Otherwise, this is maybe due to Strix Halo only having 16 PCIe gen 4 lanes, while in my mentioned comment above, building on a mini-ITX motherboard, would have 24 PCIe gen 4 lanes, which is an another advantage.
Title: Re: Framework Desktop review: Mini PC wrapped in a mini-ITX body
Post by: Dmitrii on September 20, 2025, 23:40:10
Doesn't it bother you that you installed the cooler BACKWARDS? THIS STICKER SHOULD BE ON THE BOTTOM, - THE FAN SHOULD PUSH AIR IN, NOT SUCK IT OUT
Title: Re: Framework Desktop review: Mini PC wrapped in a mini-ITX body
Post by: Adam1 on September 22, 2025, 10:58:31
I don't underatand cons.

Cons
- smaller and lighter alternatives are available with the same CPU
- RZ717 WLAN does not support 320 MHz
- RAM not user-upgradeable
- no 10 Gbps RJ-45 option

Smaller and lighter means louder and hotter. I have one miniPC and it is too small and noisy. 2-3 cm bigger and the problems go away and it will still be a miniPC.

Does a miniPC really need such a wifi card? Wifi card should be optional.

RAM is not user upgradeable but it certainly has its advantages. I would consider it a disadvantage on a regular PC but not on a PC like this.

it has 5 Gbps. Is the chance that someone will need faster LAN connection?
Title: Re: Framework Desktop review: Mini PC wrapped in a mini-ITX body
Post by: Running AI LLMs locally on September 22, 2025, 13:54:20
Ok, but why/who would need 64 GB or 128 GB RAM all of a sudden?
The latest hype is AI/running LLMs. Download e.g. LM Studio (wrapper for llama.cpp) and try it out.
The difference to ChatGPT is that it is private (and, to be fair, not as good as the best that proprietary AI has to offer, but it's catching up quick (and if the open-weight LLMs are always ~1 year behind, does it matter? (the difference is actually getting closer: as per artificialanalysis.ai/?intelligence-tab=openWeights report)).

You can also download llama.cpp directly and the LLMs can be downloaded from huggingface.co.

Current SOTA model that fit into 64 GB RAM at q8 / 8-bit quant:
huggingface.co/unsloth/Qwen3-30B-A3B-Instruct-2507-GGUF
huggingface.co/unsloth/Qwen3-Coder-30B-A3B-Instruct-GGUF

Current SOTA models that fit into 128 GB RAM (or fit using a decent quant, not the full FP16) are e.g.:
huggingface.co/unsloth/GLM-4.5-Air-GGUF
huggingface.co/unsloth/gpt-oss-120b-GGUF (from the makers of ChatGPT)
Qwen3-Next-80B-A3B (github.com/ggml-org/llama.cpp issue: "Feature Request: Qwen3-Next support")

and many more, including thinking variants, like:
huggingface.co/Qwen/Qwen3-30B-A3B-Thinking-2507.

For open weights LLMs evaluations/benchmarks, see e.g. artificialanalysis.ai/?intelligence-tab=openWeights.

NOTEBOOKCHECK, maybe write an article to explain to your readers about the AI LLM self-hosting ability or why 64 GB RAM or even 128 GB RAM are a thing all of a sudden now, because normally, 32 GB RAM are enough for most people.
Title: Re: Framework Desktop review: Mini PC wrapped in a mini-ITX body
Post by: Ed Zitron on September 22, 2025, 19:38:41
@Running AI LLMs locally:

Have you found a real use case for LLMs yet though? Besides basic chatgpt stuff like entering words in a prompt to summarize articles?
Title: Re: Framework Desktop review: Mini PC wrapped in a mini-ITX body
Post by: Running AI LLMs locally on September 24, 2025, 12:11:45
Ed Zitron, define "real use case". I don't summarize articles, but I found many use cases(, but nothing that either needs Strix Halo's 256 GB/s (practical 210 GB/s) aka the 2.5x token per second speed improvement vs just upgrading RAM in one's PC that one already has, which is much cheaper, or nothing, where Strix Halo's 128 GB RAM are enough, that's why I posted the (DIY) mini-ITX build suggestion above). A week ago OPENAI posted a LLM usage breakdown (image), see what people use LLMs for.
Title: Re: Framework Desktop review: Mini PC wrapped in a mini-ITX body
Post by: Worgarthe on September 24, 2025, 12:57:58
Quote from: Ed Zitron on September 22, 2025, 19:38:41Have you found a real use case for LLMs yet though? Besides basic chatgpt stuff like entering words in a prompt to summarize articles?
Running visual LLM locally to fully replace Google Photos with your own personal and private cloud.
Title: Re: Framework Desktop review: Mini PC wrapped in a mini-ITX body
Post by: Running AI LLMs locally on October 06, 2025, 09:04:58
Quotebuilding on a mini-ITX motherboard, would have 24 PCIe gen 4 lanes, which is an another advantage.

Just know that some non-normal desktop CPUs have only 16 PCIe lanes (like the 8400F), instead of the full 24.
Title: Re: Framework Desktop review: Mini PC wrapped in a mini-ITX body
Post by: heffeque on October 11, 2025, 01:40:14
Wonder if Notebookcheck team will re-review the temperature and noise parts, taking into consideration that they put the CPU fan upside down 🤦🏻 (a corrected picture would also be nice, so as not to confuse potential buyers).

It's not as if it's extremely clear on their instructions how it should be put together: https://guides.frame.work/Guide/CPU+Fan/489?lang=en
Title: Re: Framework Desktop review: Mini PC wrapped in a mini-ITX body
Post by: a loan, unit gone on October 11, 2025, 09:03:08
heffeque, I asked AIs (through lmarena.ai) and they say the temp increase is going to be from 5 to 20 °C.

Don't expect a fixed retest:
QuoteThe test sample was provided to the author as a loan by the manufacturer or retailer for the purpose of this review.