The 512 GB Myth in Local AI

Somewhere around one in the morning, the model card stops making sense. You started with a simple plan: run a genuinely capable model at home, no API bill, no data leaving the machine, no terms of service between you and a million-token context window. Then you found the number everyone repeats. 512 GB. Get that much fast memory, the internet says, and you are set.

You are not set. You are at the beginning of a much more interesting problem.

The video that finally explains this properly walks through the arithmetic with real hardware, real prices and real benchmarks. Its conclusion is uncomfortable for anyone with a shopping cart open: the number on the box is almost never the number that matters.

Where the 512 came from

On 15 October Meta released Mu Spark Open, an 816 billion parameter model. Mark Zuckerberg said it runs great on the new Mac Studios. The model card was more precise. It specified two Mac Studios with the M5 Ultra, and specifically the 512 GB versions. Apple has not put that configuration on sale yet. The store says late October, with no price listed.

The other frontier open models land in the same neighbourhood. DeepSeek V4.1 Flash is a 510 GB download. The popular 4-bit build of GLM 5.3 is 467 GB, which is what quantization buys you in the first place. So the logic writes itself: find 512 GB of fast memory, download the file, run the model.

That logic is wrong in a specific and expensive way. One widely shared write-up described a four-DGX Spark DeepSeek build with “a memory footprint of 476 GB”. A commenter pointed out the obvious: 476 is the download size in binary gigabytes, the number your operating system shows for a file, not what the model consumes once a conversation is actually running.

The number nobody prices

Every token the model reads leaves notes behind in memory so that later parts of the conversation can refer back to earlier parts. Engineers call this the KV cache. It lives in the same fast memory as the model weights, and it grows with every token you feed in.

That changes the question. It is not only whether the model fits. It is how much memory each token of context costs, and the answer varies by a factor of a hundred between models built on different attention designs.

DeepSeek V4.1 Flash was engineered to keep that number tiny. Its model card puts the cache at 890 bytes per token, which means a full million tokens of context costs less than 1 GB.

Mu Spark Open keeps full notes in 16 of its 62 layers and compressed notes in the rest. That works out to roughly 64 KB per token, so a million tokens costs about 69 GB on top of the weights.

GLM 5.3 keeps compressed notes across all 78 layers. At 16-bit precision that is roughly 88 KB per token, or about 94 GB for a million tokens.

Same context window. Less than 1 GB, 69 GB, 94 GB. The gap between the cheapest and the most expensive is the entire argument of the video, and it is why buying 512 GB for GLM at full context ends in disappointment.

There is a second trick that changes the shopping list even more. DeepSeek's 510 GB download is not one solid file. It splits into 552 billion parameters of main model and 196 billion parameters in a separate lookup table called engram, which stores common word sequences. For each token only a few rows of that table are needed.

An inference engine called Dwarf Star, originally written by Salvatore Sanfilippo, the creator of Redis, reads those rows directly from disk on demand. Mia AI Labs, which published a recipe for running DeepSeek across four DGX Sparks, puts all 189 GB of engram on SSDs and says explicitly: never move it into RAM.

Strip out engram and Dwarf Star's 4-bit build has 294 GB of main weights. Add a million tokens of context and you are at roughly 295 GB. The model everyone is buying 512 GB for needs closer to 300.

The hardware ladder, priced honestly

Apple's Mac Studio with the M5 Ultra reads memory at 1.2 TB per second. The 512 GB option comes only with the faster 80-core GPU, a chip that starts at 6,799 dollars with 96 GB. Apple charges 4,000 dollars to go from 96 to 256 GB, so if that curve holds, 512 GB lands near 17,200 dollars. That is an estimate, not a price, because Apple has not announced it.

There is also a detail absent from the spec sheet. By default macOS lets the GPU use about three quarters of total memory. On a 512 GB machine that is 384 GB. GLM 5.3 at 435 GB and Mu Spark Open at 471 GB do not load at all without a terminal command that raises the limit. The limit resets on every reboot, which makes the whole setup one accidental restart away from being broken.

DeepSeek's 294 GB fits with room to spare, so for DeepSeek the Mac is clean.

Two 256 GB M5 Ultras joined by Thunderbolt 5 can split one model between them. That is 18,998 to 21,598 dollars for the pair, and it is the only orderable 512 GB configuration today. Speed is the open question. On the previous generation, an M3 Ultra with 512 GB read a 62,000 token prompt at 813 tokens per second. At that rate a million-token prompt takes about 22 minutes before the first output word. Prompt caching helps enormously, one owner's 256 GB M5 Ultra took 21 seconds for a 16,000 token conversation the first time and three seconds afterwards, but you pay the long read again every time the context changes.

Four DGX Sparks, 128 GB each on Nvidia's GB10 chip, are the setup actually running DeepSeek at full context today. Nvidia raised the price to 6,950 dollars per unit in early October, so four boxes are 27,800 dollars, plus a switch that was out of stock at Nvidia and selling for 7,000 to 9,000 dollars from partners. Mia AI Lab's recipe puts about 77 GB of weights on each box and keeps roughly 40 GB free, with the context window at a full million tokens and a cache pool of 8 million tokens, enough for eight full-length sessions at once.

The benchmarks are 87.7 tokens per second for a single user, about 5,000 tokens per second reading prompts, and 342.7 tokens per second across 16 simultaneous users. A million-token prompt clears in about three and a half minutes, because this is a prefill problem long before it is a decoding one.

Four RTX Pro 6000 Max-Q cards at 96 GB each give 384 GB, less than 512, and for DeepSeek that is irrelevant because engram lives on the SSD. Local Inference Lab measured 249.1 tokens per second for one user and 17,982 tokens per second reading a 32,000 token prompt. A million token prompt clears in about a minute. The catch is price: the cards launched at 8,565 dollars and have been listed at 16,000 each since August, so four of them are 64,000 dollars before the workstation that holds them.

At the bottom of the market, two Strix Halo mini PCs with 128 GB each cost 7,758 dollars for 256 GB and run DeepSeek at 2-bit at 14 to 15 tokens per second, with prompt reads around 430 tokens per second, or 41 minutes for a million tokens. A 192 GB Gorgon Halo at 7,399 dollars may hold the full 2-bit file in a single box. Sixteen used MI50 cards, 32 GB each, bought for 180 to 460 dollars per card, give 512 GB for roughly 2,900 to 7,400 dollars and run GLM 5.2 at 12.2 tokens per second. A million tokens takes about an hour and a half.

Where 512 stops being enough

GLM 5.3's 4-bit weights are 435 GB. A million tokens of 16-bit context notes add 88 GB. That is 523 GB, more than 512 before the inference engine takes a byte.

Mu Spark Open is worse: 471 GB of weights plus 69 GB of context is 535 GB, which does not fit in one 512 GB Mac. That is exactly why Meta's own launch required two machines.

Kimi K3 is a 1.56 TB download, three times the budget, and not in this conversation at all.

Owners respond by compressing the cache. On four Sparks, GLM 5.2 users squeezed 32,000 tokens at 16-bit notes, 64,000 at 8-bit and 100,000 at 4-bit. On Terminal Bench 2.1 at 100,000 tokens they scored 70.8 percent where the full model at full precision scores 81. The top comment on that post was blunt: do not compress the KV cache. The benchmark mixes compressed weights, compressed notes and a shorter context, so it is hard to isolate which factor costs what. The score is still lower.

There is also a cautionary tale from someone who ran eight RTX Pro 6000 cards, 768 GB of VRAM, specifically to load GLM 5.2. The result was 48 tokens per second and 49,600 tokens of context, with the machine using 738 of its 784 GB. Even 768 GB did not buy a long context, because the cache problem scales with the model, not with your willingness to spend.

All local, and why the maths usually says no

If your code or your client's data genuinely cannot leave the building, none of the hosted pricing applies. That is the real case for owning this hardware. One owner of four Sparks runs complex automation for three businesses, which is a legitimate reason to want something private, secure and on premises.

For everyone else, the arithmetic is sobering. DeepSeek's API charges 15 cents per million tokens read, 60 cents per million written, and about a third of a cent per million when reading from cache. As of 15 October, OpenCode made its DeepSeek access free.

Run four Sparks flat out for a single user at 87.7 tokens per second and they write about 2.77 billion tokens a year. At DeepSeek's pricing that is 1,659 dollars of API cost. The 27,800 dollar hardware takes almost 17 years to pay back, while burning around 1,122 dollars of electricity a year at average rates. Fill all 16 sessions and run them constantly and the same year of tokens costs about 6,485 dollars through the API, a payback period of 4.3 years.

The hardware earns its keep when it is private and busy at the same time. One local session a day for personal experiments is dramatically cheaper through an API. Sixteen sessions of automated business workflows that cannot touch the cloud is where the box starts to make sense.

If any of those terms are new, the full AI glossary covers the vocabulary used here, from KV cache and context windows to quantization and throughput.

Which model, not which box

This is where the conversation should start, and almost never does.

We have already been through the middleweight division in our guide to running Qwen 3.8 locally, where the question was whether the 27B, the Flash Next or an uncensored build fits your GPU and your tolerance for guardrails. That is the sane default for most smart home builders: a 27B model on 32 GB of VRAM answers in milliseconds, runs on hardware you already own and keeps your camera feeds and automation logs inside the house.

The 512 GB class is a different sport. It is not about which model is nicest. It is about what you are legally or commercially unable to send to an API, and whether you can keep the machine busy enough to justify it.

The decision framework that actually works runs in one direction. Pick the model first. Look up its KV cache cost per token. Only then price the machine.

For DeepSeek V4.1 Flash you do not need 512 GB of fast memory. Four Sparks run it at full context today. Four RTX Pro 6000 cards run it faster. Renting those four cards costs less than 10 dollars an hour, with engram living on the SSD the whole time, which is a sensible way to test the workflow before buying anything.

For GLM 5.3, 512 GB buys you the model or the context, not both comfortably. Consider the 320 billion parameter GLM 5.3 Flash instead, which is smaller and faster and fits better, or accept a shorter context window if you need the full model.

For Mu Spark Open, use the API until the hardware prices come down or the 512 GB Mac ships with a real benchmark attached.

And if the Mac is your plan, wait for late October. Two numbers are missing: the actual price, and a real prompt-processing rate on the 512 GB version. Without both, the decision is a guess dressed as a specification.

The one question worth memorising is the one that separates people who have done the maths from people who have read a headline: what is the KV cache in bytes per token? If the answer is a shrug, the calculation is not finished.

Price the context before you price the box. At this scale, the model decides whether a million tokens costs you a gigabyte or the whole machine.

Sources

• Video: 512 GB of memory for local AI, hardware, costs and full-context benchmarks, October 2026
• Meta Mu Spark Open model card and launch notes, 15 October 2026
• DeepSeek V4.1 Flash model card, KV cache and engram specifications
• Mia AI Labs, four DGX Spark deployment recipe for DeepSeek
• Local Inference Lab, RTX Pro 6000 benchmark results
• Dwarf Star inference engine, disk-resident engram lookups
• Terminal Bench 2.1 results for compressed KV cache configurations