QuoteFramework Desktop Ryzen AI Max
AMD Ryzen AI Max+ 395, Radeon 8060S
123082 MB/s
Geekom A9 Max, AI 9 HX 370
AMD Ryzen AI 9 HX 370, Radeon 890M
86541 MB/s
HX 370: 128-bit * 5600 MT/s / 1000 / 8 = 89.6 GB/s (does check out with the measurement)
AI Max+ 395: 256-bit * 8000 MT/s / 1000 / 8 = 256 GB/s (does
NOT check out with the measurement)
Can you measure with the newest version 8.0?
QuoteUnfortunately, the system is not compatible with the recent AMD FSR4 update as the GPU is based on the same RDNA 3.5 architecture as the Radeon 890M and not the Radeon RX 9000 RDNA4 series.
Imagine paying 1600 bucks for last gen GPU architecture.
LLMsThe "Strix Halo" APU is a 256-bit chip with a theoretical memory bandwidth of 256 GB/s (256-bit * 8000 MT/s / 1000 / 8) (and ~210 GB/s practically (expected)), comparable to an entry level quad-channel (4 * 64-bit) workstation' memory bandwidth. A normal desktop PC is dual-channel at best. AMD specifically advertises "Strix Halo" for running/inferencing LLMs. You can run the same LLMs on any PC, if you have at least the same amount of RAM (well, running off of a SSD will also work, but the speed will be super slow), ATX sized or not, dual-channel RAM or not, the differences are:
- The size: This is 4.5 Liters (with integrated power supply).
- The RAM speed at which any LLM will be running at: Strix Halo is a quad-channel chip at 8000 MT/s vs a normal PC, which is dual-channel at 5600 MT/s to 6200 MT/s (2*64-bit*6200/1000/8 = 99,2 GB/s)). A (mini-)PC based on the "Strix Halo" APU will run a LLM about 2.5 times faster: 256 GB/s / 99,2 GB/s = ~2.58.
- The RAM upgradability: The LPDDR5X RAM in "Strix Halo"-based PCs is not upgradable, maybe because it runs at 8000 MT/s vs 5600 MT/s to 6200 MT/s typically seen in DDR5 UDIMMs. A DDR5 UDIMM version with upgradable RAM may appear later, but it's not going to be 8000 MT/s, like the soldered ones.
Using the relatively expensive Strix Halo APU/chip, but giving it only 64 GB RAM is wasting expensive silicon, because it's simply not enough for many LLMs (btw: the memory bandwidth will be the same, they are just using less dense RAM chips): Give it at least 96 GB RAM or just the full 128 GB, because on Windows, only 75 % can be allocated to a LLM (as far as I know, maybe it has changed) (that's 96 GB out of 128 GB RAM total).
Questions to ask yourself:
- Is the LLM speed difference of 2.5 times (150 %) and the price worth it vs simply getting 2x48GB RAM sticks or 2x64GB RAM sticks for a fraction of the price and having then more RAM (although, yes, 2x slower) vs paying 1600 bucks and being stuck with the hardware and no upgrade path?
- And, if the size matters, you can still get a mini-ITX case, AM5 mini-ITX motherboard and build a PC of the same size (or get a pre-built mini-ITX PC), with the possibility to:
- Upgrade the RAM.
- Having a dedicated GPU. There are not many choices for 4.0 - 4.5 Liter mini-ITX builds, mostly low profile RTX 4060 or RTX 5060), but this is still better and faster (and harder, stronger, hehe) than the built-in iGPU in Strix Halo:
- You get the ability to upgrade the GPU later, like when/if in 2026 the Refresh GPUs come out, using 3GB, instead of 2GB, GDDR7 chips and you get 50 % more VRAM in the same size.
- A dedicated GPU (4060 / 5060) will also have faster prompt processing (pp).
- The ability to partially or fully offload to the fast VRAM of the GPU (5060: 448 GB/s).
- A dedicated GPU adds additional capacity to the RAM.
- And, not LLM related, but: You can also game with higher FPS if you add a GPU that is faster than Strix Halo's iGPU (between RTX 4060 Laptop (=RTX 4050 desktop, which doesn't even exist, this is how bad it would be (en.wikipedia.org/wiki/GeForce_RTX_40_series)) and RTX 4070 Laptop (=RTX 4060 desktop)).
- And, if looks matter, there are many arguably better looking mini-ITX cases, too.
Love how silent it is (especially with the Noctua printed mod), yet powerful enough for stuff that my previous PC couldn't handle.
It's perfect for my HTPC setup. Really happy camper.
So odd to have a box this relatively big for nothing but a hefty cooler and PSU - there's no room for extra storage or cards, you get two m.2 slots and that's it.
Definitely not a great approach, despite Apple doing this all-soldered thing for years now.