Two of these 5080 24 GB VRAM in tensor parallel mode will give 48 GB VRAM (a +50%) at same 1.8 TB/s as a 5090. (not too long ago llama.cpp introduced tensor parallel mode)
For highest tensor parallel performance (reddit.com/r/LocalLLM/comments/1uqbk9k/comment/owof7gx) use a motherboard that has built-in bifurcation of the main PCIe 5.0 x16 slot into x8/x8 (geizhals.eu/?cat=mainboards&xf=2833_AMD+AM5~2873_ab+2&pagesize=30&promode=true).
TDP Watt per GB- 5090: 575W/32GB = 18 Watt/GB
-x% vs a 5090 = -(1-((WattTDP/GB)/
18))*100:
- 5060 Ti 16G: 180W/16GB = 11.25 Watt/GB (-37.5%)
- 5070 Ti Super: ~320/24GB = 13.3 Watt/GB (-26%)
- 5080 Super: ~380W/24GB = 15.8 Watt/GB (-12%)
- RTX PRO 4500 Blackwell: 200W/32GB = 6.25 Watt/GB (-65%) (or in other words, for the same 32 GB VRAM, the 5090 has a 2.875 times (=(575/32)/(200/32)) higher TDP (yes, the 5090 is also faster, but this is a different metric))
For AI, I personally would consider AMD GPUs again when they switch to GDDR7 (makes the token generation 30% faster at the same memory bit bus width and Watt TDP, e.g. 5080's 256-bit GPUs are at 0.9 TB/s, while AMD's GDDR6 9070 XT 256-bit GPUs are at 0.67 TB/s) as well.
English
Español
Français
Italiano
Nederlands
Polski
Português
Русский
Türkçe
Svenska
Chinese
Magyar