Why not use GDDR7, which offers 3 GB per chip density, then this GPU could have:
256-bit/32-bit per chip = 8 chips, 8 chips * 3 GB per chip = 24 GB VRAM, 24 GB VRAM * 2 (chips on both sides of the PCB) = 48 GB VRAM and the memory bandwidth would also be 30% higher, because it's GDDR7 and not GDDR6.
Alternative calculation: 32 GB VRAM * 1.5 (3 GB per chip, instead of the current 2 GB) = 48 GB VRAM.
Frankly, when it comes to AI/LLMs, I'm not interested in 32 GB VRAM GPUs..and I said this over 1 year ago.
Let's see if NVIDIA gives us consumer 48 GB VRAM GPUs in the RTX 60 series (probably not, but NVIDIA gave us RTX PRO 6000 Blackwell GPU with 96 GB VRAM, which I also didn't expect).
QuoteThe ECC memory is essential here, as a single bit flip could ruin a long render or an AI training run.
This GPU is not capable to train a LLM big enough where ECC would matter (even with several and days run, it is unlikely that something would happen). Neither is ECC necessary for fine-tuning with this GPU. Also, there are checkpoints, one doesn't lose everything, and even if, on this GPU the costs of losing a run are a few bucks a few days..in this ballpark. How I know? I asked the big LLMs over at arena.ai (aka lmarena.ai).
(en.wikipedia.org/wiki/GeForce_RTX_50_series#Desktop, en.wikipedia.org/wiki/Intel_Arc#Workstation_2)
- RTX 5090: 1792 GB/s = 512-bit * 28 Gb/s / 8.
- B70 Pro / B65 Pro: 608 GB/s = 256-bit * 19 Gb/s / 8.
So a consumer 5090 has a 3 times faster memory bandwidth and CUDA, of course. The only thing these B70/B65 could have go for them is having 48 GB VRAM and people trying to make them work without CUDA.
For inferencing these are probably fine (I know NV works using Vulkan just fine (a bit slower than using CUDA)), but then again, a 5090 has 3x the token generation and also much higher prompt processing.
5090s are going for almost $3500 USD. Unless this card costs $1200+ I don't see how this is even a comparison in your head.