2 * H100 NVL ergeben 188 GB HBM3 VRAM und 7,8 GB/s für LLM mit 175 Milliarden Parametern laut

www.nvidia.com/en-us/data-center/h100/

Nach dem Preis darf man besser nicht fragen:)

@dbbloke:

I believe they've. Isn't called the H100?

Nvidia have always skimped on Vram, historically and AMD can't really compete with Nvidia so they basically do the same thing (might release a card with few GB more vram but that's about it, nothing major)

dGPU's have never really had large amounts of memory come to think of it. It's usually iGPU's that are paired with more. But most x86 igpu's kinda suck, so might be better off just getting a Mac instead.

Why don't Nvidia do an LLM card
RTX4070 + 96gb of RAM
Ram cost is what $300 or less
and half decent RAM will do, just 80gb+
Ideally with 2 card connect option

If Nvidia don't do this, someone will! AMD??