DeepSeek V4 Flash is a 284-billion-parameter open model, and llama-modchip, a small Apache-2.0 project on GitHub, says it now runs on a 64 GB desktop with no graphics card. The file is 56.56 GiB, squeezed to 1.71 bits per weight. File size is arithmetic, so I checked it. Speed and quality are the author’s to prove.
The two loud objections in the Hacker News thread are memory and whether a sub-2-bit file is still the model. Short version: the size is real, the speed is a ceiling, quality is open.
I’m the CTO at Trellis, where we build AI tools for Amazon sellers, so my question is a small-team one: does a 64 GB box change what I keep in-house? Bon, the sum.
Does DeepSeek V4 Flash fit in 64 GB? The sum
The README gives 284.33 billion total parameters and 13.17 billion active per token. Unsloth’s GGUF header for the same model reads 284,334,567,511, which agrees to the second decimal, and the card for the 0731 release is MIT. The patches are Apache-2.0, tag v2.4.0.
Now the file. 284.33 billion parameters times 1.71 bits is 486.2 billion bits. Divide by eight and you get 60.78 GB; divide by 1.0737 for binary gigabytes and it is 56.60 GiB.
The README says 56.56 GiB, which it also writes as 60.73 GB. My figure is 0.04 GiB higher, about 43 MB, and that is rounding: 60.73 GB is 1.7088 bits per weight, which the repo rounds to 1.71. Size and bits agree.
Memory is a second question. The README says a 64 GB Linux box has about 62.5 GiB usable, and its headline row peaks at 57.61 GiB resident at 256K context. Its own tables and FAQ give 58.9 GiB for that run, which leaves 3.6 GiB instead of 4.89, and the quick-start command runs 32K context, not 256K. Tight either way.
The file it claims to beat is not a 1-bit file
The repo says it shrinks a 76.87 GiB build. I opened Unsloth’s DeepSeek-V4-Flash-0731 GGUF repo and summed the UD-IQ1_S files: 76.87 GiB, 82.54 GB, so the comparison matches the listing. One commenter said the repo mixed up IQ2_XXS and IQ1_S, and the repo contradicts itself. The README names UD-IQ1_S; the FAQ still says its output came from a “4-shard 76.87 GiB IQ2_XXS source”. Unsloth has no such file. Its IQ2_XXS is three shards at 84.62 GiB, its IQ1_S three at 76.87 GiB. The size is IQ1_S and the FAQ label is wrong.
That “1-bit” file is 2.32 bits per weight once you divide by the parameters; Unsloth’s 2-bit UD-Q2_K_XL is 90.18 GiB, or 2.72 bits. The name stops mattering.
The 1.71 file is a squeeze of a squeeze: its base is that IQ1_S file, 20.31 GiB smaller, a cut of 26.4%. Routed experts stay mostly at IQ1_S, 1.5625 bits; only the MLA projection and the attention output move to a 3.75-bit Fractal-64, while embeddings and shared experts stay at 4 to 5 bits. Whatever IQ1_S lost, this starts from there.
128 GB or 150 GB: both commenters are right
One commenter says 128 GB of unified memory already runs this fast; another says it needs about 150 GB.
First, the base file. llama-modchip and the Unsloth quants are built from DeepSeek-V4-Flash-0731, whose card calls it the official release superseding the April preview. Its safetensors sum to 166.89 GB, 155.43 GiB, or 4.70 bits per weight. That is the “about 150 GB”, and the 1.71 file is 36.4% of it. Unsloth’s own full build, UD-Q8_K_XL, is 150.75 GiB at 4.55 bits, and against that the share is 37.5%.
In Unsloth’s listing the 3-bit UD-IQ3_XXS is 97.05 GiB, which fits a 128 GB machine with about 31 GiB to spare, because RAM is sold in binary units and 128 GB means 128 GiB. The 4-bit UD-IQ4_XS is 127.28 GiB, which leaves nothing for the cache and the system. So 128 GB runs a 3-bit file and 150 GB runs the original, and nobody lied.
A Hacker News reader will add in the first hour that 0731 is two releases old: V4.1-Flash in September, V4.1-Flash-1017 in October. This squeeze is of a superseded checkpoint, worth knowing before a 57 GiB download.
The other objection is quality: below 4 bits, models get incoherent, though people argue both ways and some call a 2-bit DeepSeek useful. The original is only 4.70 bits, so 1.71 keeps 36.4% of its size. Can the answers survive? Not settled here, and the repo does not settle it.
Look at what its quality columns compare. Cosine 1.00000 and KL 0.0000 compare the patched runtime with the unpatched one on the same 56.56 GiB file, per the README’s own note: the new kernels do not change the answer. That says nothing about whether the file matches the original weights. I searched the README, the deep-dive and the FAQ for a perplexity or benchmark score against the original model, and there is none.
What 9 tokens a second really means
Decode is 9.1 to 14.7 tokens a second, prefill 44.36. Every token reads the active weights once. At 1.71 bits, 13.17 billion active parameters are 2.82 GB per token; at 9.06 tokens a second that is 25.5 GB a second. A dual-channel DDR5-5600 board peaks at 89.6 GB a second on paper (5,600 million transfers, 8 bytes, 2 channels), so the claim is not absurd. Attention weights sit at higher bit widths, and the README’s own profile puts 61.81 of 110.3 milliseconds in attention and shared-expert products.
Plain decode is 9.06 to 9.95 tokens a second. The 14.7 comes with a drafting trick that guesses from text already written, and it pays off when the model repeats itself, like refactoring code. For writing something new, plan on 9.
Those speeds are the author’s own, on a single 16-core Ryzen 9 9955HX, and the seven community benchmark issues in the repo (#5, #6, #8, #9, #10, #12, #14) all ran a 30B Nemotron model, not this one.
What do 44 and 9 buy? A 4,000-token prompt takes 90 seconds before the first word, a 20,000-token prompt 451 seconds, 7.5 minutes, then a 500-token answer another 55 seconds. Those are floors. 44.36 a second is the repo’s pp512 peak, its longest cold prefill is 580 tokens at 43.98, and its multi-turn table shows 38.44 by 2,418 tokens of context. A 20,000-token prompt is 34 times the longest anyone measured, so expect worse. That is not a chat window.
What I would use it for, and what I would refuse
A batch job: rewrite 1,000 product-listing blurbs, 800 tokens in and 300 out each. On his numbers that is 18 seconds of prefill and 33 of decode per item, 51 seconds, 14.2 hours per 1,000, about 1,690 a day. An overnight queue on a box with no GPU and no API bill, and the data never leaves the building.
So: work that can wait and can be checked. Sorting reviews into complaint types. Pulling attributes from messy listing text. Drafting rewrites a person or a script reads before anything goes live. A wrong answer costs a re-run, not a customer.
I would refuse it for three things. Anything interactive, because 90 seconds before the first word is a dead tool. Agents with long tool prompts, because every turn pays the prefill. And anything where a silent wrong answer costs money: bid changes, price changes, compliance claims. Until someone scores it against the original weights, I would not let it near money.
For a small team, what changes is where the privacy line sits: a 64 GB box without a GPU can hold a 284B mixture-of-experts model. What does not change is the price of speed, and you still need your own test set to know if it is good enough. I looked at the same trade, a huge MoE held in system RAM, in my post on MoE expert offloading, and the catch there was one user at a time.
The repo has a benchmark-submission template, and every scorecard I opened was for the 30B model. The day a stranger posts a 284B scorecard from their own machine, I’ll redo this sum with their speed.
