Nvidia DGX Spark vs AMD Strix Halo: $4,000 AI supercomputer challenged by $2,199 mini PC with better real-time performance

[Redaktion]

GMKtec's AMD Strix Halo-powered EVO-X2 mini PC is taking aim at Nvidia's $4,000 DGX Spark, claiming faster AI response times at nearly half the price. The test raises questions about value versus raw throughput.

https://www.notebookcheck.net/Nvidia-DGX-Spark-vs-AMD-Strix-Halo-4-000-AI-supercomputer-challenged-by-2-199-mini-PC-with-better-real-time-performance.1159829.0.html

[A]

While GMKtec's test shows Nvidia's setup remains the hardware to beat for large-model, high-throughput operations

Where? The largest model they tested was the 70B one and the Strix Halo won on generation speed, it only lost on first token which isn't really that important

The DGX has been a complete flop.

Also, I am pretty sure I saw a youtube video of this, the models tested and results seem to match that youtube video. Except it seems like they didn't even use the video itself but used the youtube commenter's data who summarized it. You notice because on the 70B model, the strix halo scored 4.97/sec, not 4.9/sec but the commenter left out the 7 at the end by accident.

Youtube video is called:

NVIDIA DGX Spark – A Non-Sponsored Review (Strix Halo Comparison, Pros & Cons)

While it is nice to have it in text format over videos, they still should source who they got their data from as otherwise it is plagerism. Or they could have just rented a DGX and do their own testing with more models and not just language models but other stuff as well

[A]

By the way, is there any plan to add AI benchmarks to the benchmarking suite of notebookcheck?

[Laitaps]

I don't have a DGS-Spark to verify this but I do have an EVO-X2 with a Max+395 128GB variant.  The GMKTek box is a good piece of hardware and performance is likely to considerably improve since RocM is not very mature for this chip.  So the gap should widen.  There are however disadvantages... 1. Aforementioned driver immaturity.  2. Support for some tasks in PyTorch due to 1.  3.  2.5 Gb Ethernet instead of 10 (supported on other strix halo machines).  4. Lack of NVLink to connect multiple machines with a dedicated high speed interconnect.  5. The AMD chip does not support some of the lower precision data types that NVidia does. 

If you are trying to gain experience with NVidia's ecosystem, the Spark may still be the better choice but it really depends on workload.

[A]

I don't have a DGS-Spark to verify this but I do have an EVO-X2 with a Max+395 128GB variant.  The GMKTek box is a good piece of hardware and performance is likely to considerably improve since RocM is not very mature for this chip.  So the gap should widen.  There are however disadvantages... 1. Aforementioned driver immaturity.  2. Support for some tasks in PyTorch due to 1.  3.  2.5 Gb Ethernet instead of 10 (supported on other strix halo machines).  4. Lack of NVLink to connect multiple machines with a dedicated high speed interconnect.  5. The AMD chip does not support some of the lower precision data types that NVidia does. 

If you are trying to gain experience with NVidia's ecosystem, the Spark may still be the better choice but it really depends on workload.

1. From my understanding, all benchmarks so far only use the GPU and don't even use the beefy NPU. So GPU+NPU should lead to quite a good increase in performance. The support should be out soon though the software has to support parallelism and the amd npu.

2. Is 10gb Ethernet that needed? These aren't servers and if one needs 10gb there is usb to ethernet. Of course there are cheaper desktops than the GMK unless it is on a deal.

3. NVLink is a waste of time for these things. Think about it, the advantages of these is being able to run large models due to good amount of relatively fast unified ram. The moment you are doing something like NVLink 2 of them, that is 8k. And for that price you are just better off with multiple gpus or going with servers. Even 1 of these at 4k would lose to dedicated gpus other than power consumption

4. If your goal is to get experience with nvidia ecosystem, just buy an old GPU. Spark uses an ARM processor so the spark takes a lot of extra work to get stuff running and some may not even work.