llama.cpp Gets a "Particle Scattering Sampler" — A New Approach to Drawing Out Diversity Without Sacrificing Generation Quality in Local LLMs

An experimental sampler called "scatter" has been added to llama.cpp. It was introduced alongside a detailed write-up on Reddit's r/LocalLLaMA.

Traditional temperature flattens the entire probability distribution, which boosts creativity but has the downside of raising probabilities even for weak tokens at the tail end (unwanted candidates). "scatter" takes a different approach: it redistributes probability only among the top candidates that have already been filtered by top-k, top-p, and similar methods. Using a Gaussian kernel based on rank distance, it swaps probabilities between candidates at nearby ranks. The result is a sampler that softens the head of the distribution without dragging unwanted tokens up from the deep tail.

On top of that, it ships with a wide range of options, including an entropy-based adaptive mode, a mechanism for absorbing repeated tokens, and adjustable stochastic firing rates (collision gate). While useful for creative text generation, it reportedly carries risks for tasks that demand strict precision, such as code generation or JSON output. The implementation is already part of the default sampler chain and behaves as a noop when no parameters are set.

MLX-serve: A New LLM Inference Server for Apple Silicon, Written in Zig

MLX-serve, an LLM inference server for Apple Silicon (M1–M4), was showcased on Hacker News. Its distinguishing trait is that, while built on the MLX framework, it is implemented in Zig.

Thanks to its unified memory architecture, Apple Silicon has the advantage of being able to load large models directly into GPU memory. MLX-serve appears to leverage this advantage while combining it with Zig's lightweight runtime characteristics to deliver an efficient inference server. Details are available on the project's published website. For macOS users who want to host LLMs locally, it looks like a promising new option.

Gemma 4 E2B Impresses on a Low-Spec PC — 9 tok/s on a Core i5-6500

A report posted on Reddit's r/LocalLLaMA describes achieving a generation speed of 9 tokens per second while running Gemma 4 E2B (a 2-billion-parameter model) on an Intel Core i5-6500 (6th generation, released in 2015).

The poster rates its output as "better than ChatGPT 3.5, and probably on par with ChatGPT 4." They previously used Qwen 3.5 4B and were impressed by that one too. As an example showing that we have entered an era where practical local LLMs run even on low-spec hardware, the post is drawing plenty of attention in the community.

Malware Techniques for Evading LLM-Based Code Scanning Spark Discussion

A technique that abuses LLM safety refusal behavior to keep LLM-based code scanning tools from detecting malware has become a topic of conversation.

Specifically, the trick appears to involve embedding content within the malware's code that triggers LLM safety filters, steering LLM-based security scanners into refusing with something like "I cannot analyze this code." As automated security auditing powered by LLMs continues to spread, this kind of evasion technique deserves attention as a new threat model. It is still at the stage of conceptual discussion for now, but it is a risk that organizations deploying LLM-based security tools should be aware of.

Criticism Mounts over the Federal Government's "AI First" Policy

An article titled "AI First: How the Federal Government Is Prioritizing AI over People and Planet" picked up 25 points on Hacker News, along with 19 comments.

The article critically examines what impact the federal government's policy of prioritizing AI infrastructure investment has on workers and the environment. It points to issues such as the power consumption of AI data centers, their water usage, and effects on the labor market. How to strike a balance between the benefits and the costs of AI-promotion policy will likely remain an important topic of debate for some time to come.

---