How do you run an ultra-massive model on local hardware? What caught the community's eye on August 2 was a string of reports on exactly these kinds of "extreme inference experiments." At the same time, debate has flared up over the price war triggered by DeepSeek's low-cost models and over how to evaluate Big Tech's AI investments.

Running the 1.56TB Kimi K3 on a CPU and 8GB of RAM

A post on Reddit's LocalLLaMA community describes an attempt to run Kimi K3, a MoE model with a 1.56TB checkpoint, on nothing but a CPU and 8GB of RAM—no GPU. The author built their own inference engine in C99, using neither BLAS nor any GPU path, and it ships as a binary of just 6 files and 176KB.

The key is a design that leans into the architecture's characteristics. 93% of Kimi K3's checkpoint consists of "routed experts," of which only 16 out of 896 fire per token. So the experts aren't kept resident in RAM; they're read on demand from NVMe and multiplied while still packed in 4-bit. Skipping the dequantization step is the point.

The result: a peak RSS of 8.24GB on the smallest preset, at roughly 33 seconds per token. Allocating around 128GB of RAM cuts that to about 20 seconds per token, which turned out to be the ceiling. The author themselves repeatedly notes that "this is not a practical way to use K3"—the goal is to understand the architecture "through implementation." It needs 1.7TB of disk for the checkpoint and is nowhere near practical speed, but as a demonstration of how to map an MoE's structure onto a compute graph, it's fascinating material.

Local Benchmarks of DeepSeek-V4-Flash-0731: Big Gains from Batch Size Tuning

Also on LocalLLaMA, a specs report came in from someone running DeepSeek-V4-Flash-0731 (the 0731 build) locally. The environment: an AMD EPYC 74F3 (24 cores), 8-channel 3200 DDR4, and an RTX A6000 48GB. Inference speed held steady at about 17.2 tokens per second, and with 48GB of VRAM the setup should theoretically be able to hold a context of up to 1 million tokens (though prompt processing speed is expected to suffer).

What stands out is the tuning payoff. Setting batch size to 8096 sent prompt processing (PP) soaring to nearly 400 tokens per second at the start, and it still maintained around 300 tokens per second at a context of 20,000. That's a big improvement over the default of roughly 70 tokens per second, and it shows that re-examining batch size can pay off in local deployments.

Local benchmarks of DeepSeek-V4-Flash itself have been reported recently as well; this round adds verification from a different user on a different hardware configuration, plus a new insight in the form of batch size tuning.

Kroma: A LoRA for Krea 2 and ComfyUI's ".diff" Weight Format

Trending on Hugging Face is "Kroma v0.1," a LoRA for the image generation model Krea 2. It ships as a single ComfyUI-compatible safetensors file (about 1.88GB), rank-reduced to rank 256 / alpha 256.

What's technically interesting is how the weights are carried. Alongside a conventional "pure" LoRA, it bundles the learned RMSNorm scales and modulation tensors as ".diff" weight deltas, and ComfyUI's LoraLoader applies these automatically. At strength 1.0 it reproduces the original fine-tune exactly; lowering it to 0.7–0.9 is recommended if you want to dial the effect back. It works with both Krea 2 base and Turbo, and the Turbo side can be sped up with 8 steps and CFG around 0.

Model Commoditization and the Price War DeepSeek Is Fueling

One story drawing attention in the community is Axios's piece "AI Models as Commodities." It appears to cover the AI price war brewing around DeepSeek's low-priced models, but since the source article's text wasn't included in the collected data, I'll hold off on the details. Even so, the picture is clear: DeepSeek's low-cost models are pushing everyone's pricing downward, and the "commoditization" debate—cost competition taking the foreground over model differentiation—is heating up.

In the same vein, while Futurism reported that "Meta has almost nothing to show for AI," a rebuttal arguing that the numbers tell a different story also drew attention. How to judge the returns on Big Tech's enormous AI investments looks set to remain a divisive question.

A Minor Point of Debate Around AI, Copyright, and Credit

On Hacker News, an article titled "Don't credit the LLM" also made the rounds. It appears to ask how an LLM's contribution to generated output should be treated as copyright credit, but here again the source text wasn't collected, so the details of the author's position can't be pinned down. With lawsuits and rulings piling up around generative AI copyright, how far this line of argument spreads is worth watching.