The practical bar for local LLMs has been raised another notch. A new checkpoint trending on Hugging Face runs a 35-billion-parameter-class model on a smartphone-class amount of memory. On the hardware side, meanwhile, a structural wall is becoming visible: memory bandwidth does not scale the way compute does. Drawing on this week's collected items, this post rounds up interesting trends on both the implementation and research fronts.
A 35B MoE on Smartphone-Class Memory — Edge0-35B-A3B Preview Released
"Edge0-35B-A3B-preview" is an early preview that aims to run a 35B MoE (mixture-of-experts) model at practical speeds in under 3GiB of active memory. The core of the trick: the entire 4-bit quantized checkpoint stays on storage, and only the routed experts are streamed into RAM on demand. Because memory usage is determined by "the set of experts actually used" rather than the parameter count, no upfront weights download or sharding is required.
Speed projections are given too: 15 tokens/second for decoding, and 140 tokens/second for long-prompt processing (prefill). To tackle the problem of inference stalling while experts load, the system overlaps weight loading with the forward pass using a trained head called "prerouter" that predicts routing destinations one step ahead, reportedly securing up to a 59% improvement in decode throughput. Quality loss from quantization is compensated by a distillation method called Recover-LoRA, with the int4 version said to land within 3.9 points of the fp16 baseline.
It is still explicitly labeled an "early preview of the pipeline," so real-world evaluation is yet to come — but the design philosophy of dropping the assumption that the whole model must be loaded into RAM points squarely toward lowering the barrier to local inference. The preview-stage claims await verification in actual running environments.
Micron's Chart of the "Memory Wall" — Compute Triples, Bandwidth Falls Short of Doubling
A single chart from Micron's memory tutorial at Hot Chips 2026 has been drawing attention in the LocalLLaMA community on Reddit. Its content is a simple contrast: normalized TFLOPS from TPU v3 to R200 grow roughly 3x every two years, while bandwidth from HBM2e to HBM4 fails to even reach 2x over the same period. And because the chart is on log-log scales, the actual divergence is even larger than it looks.
What makes it interesting is that it sums up "so what is the industry doing about it?" as a three-step prescription: (1) place memory next to the compute units, (2) bring memory closer over shorter links, and (3) build compute units into the memory itself. Samsung is already shipping the third of these with LPDDR5X, reportedly measuring 3.01x tokens/second on llama 3.1 8B.
Given how easily LLM inference speed is bottlenecked by memory bandwidth, this "memory wall" connects directly to debates over benchmark scores and per-token pricing. Edge0's attempt to cope via storage streaming can be read as another way around the same wall.
LLM Judges Are Lenient With "Strong Performers" — Measuring Capability-Dependent Bias
"LLM-as-a-judge" — having LLMs evaluate other LLMs — is in wide use thanks to its low cost, but two studies questioning its reliability have appeared together.
The first measures systematic bias in absolute evaluation tasks. Analyzing 4 benchmarks and 6 models (36 judge/evaluatee combinations), it reports the following. A judge model's own task accuracy correlates strongly with its judgment accuracy (Pearson r >= 0.90) and correlates inversely with directional bias. Moreover, every judge model showed a consistent tendency to grade more capable evaluatees leniently (r >= 0.83). In other words, the structure is: "the more capable the model, the more it benefits from the judge." The team proposes calibrated majority voting (WMV), which weights multiple judges' verdicts via online estimation of false detection rates — and requiring no ground-truth labels is what makes it practical.
The second, GAUGE, focuses on evaluation gates for task-oriented agents. It takes a ubiquitous form of offline evaluation: an LLM judge scores conversations with persona-driven LLM user simulators, and high-scoring agents get adopted. Across 25 agents from 6 providers, 57.5% of conversations rated "satisfied" had in fact failed the customer's task. Satisfaction and task success rate were effectively uncorrelated, a trend consistent across 5 evaluator populations and 2 benchmarks. The gate's rankings were robust in bands where capability gaps are large, but lost discriminative power between top models of similar ability.
The more one leans on cheap automated evaluation, the more this kind of structural bias quietly does its work. The age in which a benchmark score no longer means ability outright may already have arrived.
Hallucination Has a Lower Bound — "The Limits of Compression"
Factual hallucination tends to be discussed as a "the knowledge just wasn't in the model" problem, but theoretical work showing there is more to it has also been released. "The Cost of Compression" formalizes closed-book QA as a setting in which "M training facts are compressed and stored into B bits, in a world with N possible queries and K possible answers." The derived lower bound on the error rate decomposes neatly into two terms: (1) distortion from observed facts being lost to compression, and (2) coverage shortfall on unobserved facts.
Put differently: packing facts into finite memory means even facts once seen must be stored approximately, and that degradation cannot be reduced to zero in principle. Against the wish to eliminate hallucination entirely without growing the model, this draws an information-theoretic line saying "this much is impossible." It should provide a foundation for teasing apart what is an essential limit and what is room for implementation improvement.
"Cheap Pre-Execution Checks" to Stop Runaway Agents
LLM agent research also brings countermeasures for failures that happen quietly. A paper titled "Look Before You Leap" homes in on "silent failures" among the actions agents emit (shell commands, file edits) — actions that produce the wrong effect without raising an error. It proposes a cheap and underrated form of oversight: running deterministic static verification before an action executes.
In an evaluation spanning 9,930 shell commands and 482 tools, it detected 95.8% of invalid commands at a 10.0% false-positive rate. Syntax checks and binary-existence checks were "oracle-complete," sustaining zero false positives; all false positives reportedly came from flag checks. The verifier is designed so it can abstain when uncertain — "no guesswork" is built into the design. As an agent safety measure, it looks worth trying before bolting on a heavy monitoring model.
GitHub Organizes 10 On-the-Ground Terms for the AI Era
According to Japanese media reports, GitHub has published a guide explaining AI-related terms that are spreading fast across software development workplaces. It is said to organize 10 key terms, including "FDE," "harness," and "open weights." It is an attempt to cement a shared understanding of vocabulary that gets tossed around as a matter of course — likely a help for teams vexed by terminology drift.
---
Looking across this week's collected items as a whole, the topics converged on two axes: on the research side, "the reliability of evaluation and monitoring"; on the implementation side, "how to live with memory as a physical constraint." Between the flashy model releases, it is worth noting that these grounded lines of inquiry are quietly maturing.