iLLaDA: A Masked Diffusion Language Model Closes In on Autoregression at 8B Scale

Most conventional LLMs rely on autoregressive next-token prediction with causal attention. In contrast, **iLLaDA (Improved Large Language Diffusion Architecture)**, introduced by a team including Renmin University of China, is a masked diffusion model trained from scratch at 8B parameters, using bidirectional attention consistently from pretraining all the way through SFT.

The training scale: 12 trillion tokens of pretraining, plus an instruction corpus of 25 billion tokens over 12 epochs for SFT. Validation also incorporated a GPT-4-based evaluation process, showing that the masked diffusion approach can become a credible alternative to autoregressive models.

Diffusion-based language models have long been studied as a non-autoregressive architecture, but they have struggled to match performance once scaled up. iLLaDA's results are an important milestone, suggesting that this "scaling wall" can be overcome.

A Small Number of Wikipedia Edits Shape LLM Values

"Can a small number of edits to AI training data actually change a model's outputs?" A paper that answers this question empirically is drawing attention.

The study focuses on **Pro-Animal Wikipedians (PAW)**, a volunteer group that edits Wikipedia articles on animal welfare. They have made 125 edits across 115 pages, adding animal welfare information complete with reference sites. Wikipedia is included in the training data of nearly every major LLM and is weighted more heavily than generic web-crawled text.

Using gradient-based data attribution (such as the Bergson method), the research team quantitatively evaluated the impact of these edits on LLM outputs. The results show that a small group of coordinated volunteers can genuinely change how AI systems talk about specific topics.

"Information transparency" and "manipulation resistance" in AI training data will become increasingly important points of debate. While this case is a benign example centered on animal welfare, it also points to the risks if the same techniques were applied with malicious intent.

Detectable but Not Controllable — The Geometric Gap in LLM Interpretability

Understanding the internal representations of LLMs and detecting or steering specific behaviors is a central goal of mechanistic interpretability. However, a paper from a team including the University of Bologna in Italy calls into question an assumption underlying this goal.

Much prior work has implicitly assumed that the direction that **detects** a behavior and the direction that **controls** it are the same, or at least close. The team put this hypothesis to a geometric test.

The results are telling. There is a non-negligible angular discrepancy between the direction that best detects a behavior and the direction that most effectively elicits it. In other words, being able to measure what a model "knows" is a separate problem from knowing how to manipulate it.

This finding underscores the need to treat "monitoring" and "intervention" as distinct concerns in LLM safety research. Even a highly accurate detection probe could produce unexpected side effects if repurposed directly for control.

Detecting Jailbreaks from Entropy Trajectories

In LLM security research as well, approaches that look at model internals are advancing.

The newly published "**What Intermediate Layers Know**" proposes a method for detecting jailbreak attacks by analyzing layer-by-layer trajectories of token-level prediction entropy in a frozen LLM.

While most defenses operate at the prompt or output level, this approach stands out for directly targeting the encoding of harmful intent in the model's intermediate layers. By observing entropy dynamics, it suggests the possibility of estimating risk before any output is generated.

A Wave of Agent Tools: From Sandboxed Execution to Remote Control

As AI agents move toward practical use, developer tooling is rapidly filling out.

**OpenAI Sandbox Agents** pushes agent execution into an isolated environment, simultaneously tackling two challenges: the risks of shell commands (such as accidentally running `rm -rf`) and the loss of working state when the process terminates.

Meanwhile, Hacker News has seen a flurry of tool announcements:

Hezo**: A self-hosted AI agent team that operates autonomously without ever seeing your secrets

Zedra**: A tool for remotely operating AI coding agents

Swarm-test**: A pre-production testing framework for multi-agent systems

The unease of handing an agent the shell is a shared concern among engineers thinking about production deployments. An approach that safeguards agent safety through three layers — isolation, monitoring, and testing — is steadily becoming the norm.

---