On 24. April 2026 DeepSeek-V4-Flash(-preview) came out.
Now, on the 31. August 2026, the full version has been released.
Quote from: 192 GB+24 GB VRAM is nice on July 21, 2026, 11:06:14(the current DeekSeek-V4-Flash is a preview version and the final version is supposed so be released this month)
And, as said, the full version has been released this very month indeed, both, not only on their API, but also the weights:
Quote from: reddit.com/r/LocalLLaMA/comments/1vbidkp/deepseekv4flash_has_been_updated_the_official/DeepSeek-V4-Flash has been updated, "The official release of DeepSeek-V4-Pro will follow soon"
Quote from: reddit.com/r/LocalLLaMA/comments/1vbk5ob/new_deepseek_v4flash_achieves_50_on/New DeepSeek V4-Flash achieves 50 on ArtificalAnalysis Index, 1 point below GLM-5.2 and GPT-5.6 Luna
Quote from: artificialanalysis.ai/articles/deepseek-v4-flash-0731-scores-50-on-the-artificial-analysis-intelligence-index-10-points-above-previous-deepseek-v4-flashDeepSeek V4 Flash 0731 scores 50 on the Artificial Analysis Intelligence Index, a 10-point jump over DeepSeek V4 Flash (released April 2026) that puts it 6 points ahead of DeepSeek V4 Pro
[..]
Weights from e.g. here: huggingface.co/unsloth/DeepSeek-V4-Flash-0731-GGUF (follow their guide on how to run this model locally (and therefore privately)).
On 128 GB RAM and/or VRAM systems, it's worth trying a 3-bit quant (smallest is 103 GB).
If you can't fit the full 162 GB weights or any of the, still usable, 3-bit quants, into your RAM + VRAM, remember that all you need is (fast) memory, and so the rest of the quant can be fit/stay on your SSD. A PCIe 5.0 x4 SSD achieves read speeds of about 14 GB/s and RAID cards can go well beyond 14 GB/s, but any SSD that will fit the full weights or a quant, will allow you to run it. And don't worry, since the SSD is only being read from, not written to, the flash cells are not "consumed":
Someone tested many models running from VRAM + RAM + SSD:
Quote from: reddit.com/r/LocalLLaMA/comments/1ow0jj0/running_a_1_trillion_parameter_model_on_a_pc_with/Memory mapping (mmap) in llama.cpp lets it read model files far beyond RAM capacity.
Quote from: gist.github.com/ubergarm/0681a59c3304ae06ae930ca468d9fba6No it is not swap and won't kill your SSD's read/write cycle lifetime.
Quote from: reddit.com/r/LocalLLaMA/comments/1vdbix4/deepseekv4flash_284b_on_53gb_of_memory/DeepSeek-V4-Flash 284B on 5.3GB of memory
You guessed it, since it's about memory size first and foremost, you can run DeepSeek-V4-Flash even from your phone (BigMoeOnEdge) (reddit.com/r/LocalLLaMA/comments/1vd874t/comment/p179tnm).
In all mentioned cases, the MoE architecture does the heavy lifting, since only a small part if a MoE model needs to be activated to generate a token, so the speed is still usable.
More here: notebookchat.com/index.php?topic=315954.0 ("Your own ChatGPT, offline: AI without the cloud on your laptop") and comments.
(the comment system wouldn't allow me to fit this on the same, previous, post..)