News:

Willkommen im Notebookcheck.com Forum! Hier können Sie über alle unsere Artikel und allgemein über notebookrelevante Dinge diskutieren. Viel Spass!

Main Menu

Post reply

The message has the following error or errors that must be corrected before continuing:
Warning - while you were reading 2 new replies have been posted. You may wish to review your post.
Other options
Verification:
Please leave this box empty:
Shortcuts: ALT+S post or ALT+P preview

Topic summary

Posted by DeepSeek-V4-Flash-0731
 - Today at 12:57:42
Open-weight models, that is.
Posted by DeepSeek-V4-Flash-0731
 - Today at 12:56:35
See various up to around 300B parameter AI LLM models compared:
artificialanalysis.ai/?models=qwen3-5-122b-a10b-non-reasoning%2Cqwen3-5-122b-a10b%2Cmimo-v2-5-0424%2Cqwen3-6-27b-non-reasoning%2Cminimax-m2-7%2Cnvidia-nemotron-3-super-120b-a12b%2Cqwen3-6-35b-a3b-non-reasoning%2Cstep-3-7-flash%2Cqwen3-6-35b-a3b%2Cqwen3-6-27b%2Cmotif-0714%2Chy3%2Cgemma-4-31b%2Cgemma-4-31b-non-reasoning%2Cmistral-medium-3-5%2Cgpt-oss-120b%2Cdeepseek-v4-flash%2Cinkling-small&intelligence=artificial-analysis-intelligence-index
Posted by DeepSeek-V4-Flash-0731
 - Today at 11:37:51
Quote from: 192 GB+24 GB VRAM is nice on July 21, 2026, 11:06:14(the current DeekSeek-V4-Flash is a preview version and the final version is supposed so be released this month)
And, as said, the full version has been released this very month:
Quote from: reddit.com/r/LocalLLaMA/comments/1vbidkp/deepseekv4flash_has_been_updated_the_official/DeepSeek-V4-Flash has been updated, "The official release of DeepSeek-V4-Pro will follow soon"
Quote from: reddit.com/r/LocalLLaMA/comments/1vbk5ob/new_deepseek_v4flash_achieves_50_on/New DeepSeek V4-Flash achieves 50 on ArtificalAnalysis Index, 1 point below GLM-5.2 and GPT-5.6 Luna
Ok, I don't think the weights have been released yet.

On 128 GB RAM systems, like Strix Halo (note: AFAIK, on Linux more RAM can be allocated), it may be worth trying to run a 3-bit quant (smallest is 103 GB, huggingface.co/unsloth/DeepSeek-V4-Flash-GGUF).

And the upcoming Gorgon Halo 192 GB RAM configs will fit the full DeepSeek V4 Flash weights (162 GB) + have plenty of memory left for the context on Linux, where more RAM can be allocated than on Windows (if still so), just like with Strix Halo.

Meanwhile llama.cpp DeepSeek-V4 support is coming along nicely, too:
Code Select
llama.cpp] git log --pretty=format:"%h %ad %s" --date=short -P --grep "deepseek(?:.*?[^[:alnum:]])?v?4\b|ds-?v? *?[-+_]?v?4" -i
9d9a6d29f 2026-07-31 SYCL: add oneMKL GEMM flash attention for XMX-accelerated prompt proc... (#25025)
69e62fc77 2026-07-31 llama : enforce the same K and V cache types for DeepSeek V4; enable FA if V cache is quantized (#25871)
dee2a846b 2026-07-27 ggml : adjust logic for offloading ops to weight's backend (#25832)
e8e6c7af2 2026-07-22 minor: fix reasoning preserve var for DS4 [no ci] (#25999)
f534da26e 2026-07-22 Fix DeepSeek4 crafted template (#25414)
67b9b0e7f 2026-07-22 llama-arch: fix DeepSeek4 APE tensor op (#25945)
846e991ec 2026-07-22 cuda: add sqrt_softplus in topk-moe for dsv4 (#25896)
91d2fc387 2026-07-20 llama_dsv4: write only used rows in state (#25325)
4937ca83f 2026-07-18 llama-quant : exclude i32 ffn_gate_tid2eid routing table from quantization (#25787)
5d5306bf3 2026-07-17 tests : initialize all tensors in test_dsv4_hc to avoid NaNs in sentinel tensors (#25822)
0dc74e332 2026-07-17 DeepseekV4: Add fused hyper-connection ops (#25585)
33a75f41c 2026-07-15 DeepseekV4: reduce graph splits (#25702)
7f575c39d 2026-07-14 DeepseekV4: fix seq_rm (#25588)
13f2b28b0 2026-07-11 DeepseekV4: clear cache only for seq rather than full (#25521)
00f5442cc 2026-07-11 ggml : add GGML_OP_LIGHTNING_INDEXER that implements DeepSeek V3.2/V4 lightning indexer (#24231)
2ed3c1abb 2026-07-10 llama : make all KQ masks f16 if FA is used, remove zero attention bias, remove raw_k repeats in DeepSeek V4 (#25370)
024c46ae4 2026-07-07 llama: fix quantized kv-cache for dsv4 (#25202)
8c146a836 2026-06-29 DeepSeek V4  (#24162)