Single-Neuron Edit Fixes Gemma 4's Repetition Loop — but the "Thinking Doom Loop" Remains
Gemma 4's instruction-tuned models have a reproducible bug: on long fact-listing prompts (all episodes of a TV series, the 88 constellations, the original 151 Pokémon, and so on), they fall into repetition loops. The looping rate reaches up to 95%, and neither rephrasing the prompt, switching inference engines, nor adjusting sampling parameters resolves it.
In a new paper published on arXiv, researchers discovered that the cause is localized in **just a handful of MLP neurons**. They pinpointed the culprit through layer-by-layer ablation experiments and neuron-level attribution analysis, and in the E2B model, **flipping the sign of a single neuron** was enough to eliminate the repetition loop. Larger models could be fixed similarly by suppressing a small number of routing experts, with general benchmark scores preserved.
However, this "surgery" is not a cure-all. Under longer thinking budgets, a phenomenon remains in which the model falls into a "doom loop," repeatedly self-correcting around facts it cannot recall. This is a knowledge-accuracy problem that deleting a circuit cannot solve — weight surgery can remove the loop, but it cannot fill in missing knowledge, the authors point out.
LLM-as-a-Judge Verdicts Are as Unstable as a Coin Toss
Automated LLM evaluation (LLM-as-a-Judge) is widely used for ranking model outputs, training reward models, and updating public leaderboards. Yet its run-to-run reliability had never been adequately validated.
The findings from a validation experiment are striking: using two judge models, GPT-4o-mini and GPT-4.1-mini, the researchers ran 50 pairwise trials and 50 pointwise trials per question across 29 tasks in 10 categories. **When the same evaluation is repeated, the pairwise preference verdict flips 13.6% of the time on average**. For 28% of questions the flip rate exceeded 20%, and on one question it reached 56%. GPT-4o-mini also exhibited a bias favoring the first position (72% favoring A, p = 0.024).
Pointwise score differences, meanwhile, were small — 0.19 to 0.36 points on a 10-point scale — and often not statistically significant. In other words, a contradiction arises: the scores show almost no difference, yet the pairwise evaluation still picks a winner.
The study estimates that **matching a reference verdict with 95% probability requires 11 repeated trials on average — and 15 for high-variance questions**. The conclusion: single-shot LLM judgments are far too noisy for high-stakes evaluations.
VLMs Answer Image Questions Correctly Without Looking — the Two Faces of the "Mirage" Phenomenon
Vision-language models (VLMs) confidently — and often correctly — answer image-based questions even when no image is provided. It has long been pointed out that this "mirage phenomenon" inflates benchmark scores, but new research shows that it is **not a single phenomenon but the product of two distinct mechanisms**.
Using a contrastive probing framework named "Mirage Probes," the researchers confirmed that mirage behavior is linearly recoverable from the VLM's internal activations. The analysis identified two regimes:
Text-bias regime**: the model answers from linguistic prior knowledge alone, without accessing visual representations
Spurious-image regime**: the model "fabricates" nonexistent visual content in its latent space and answers as if it had grounding
The former can be addressed by cleaning the text distribution, but the latter is rooted in the visual representations themselves, requiring intervention at the representation level. "Faithful visual grounding requires representation-level intervention," the authors argue.
Agent Traces Become a Breeding Ground for "Skill Leakage" — the RedAct Framework
A problem has been identified in which execution traces published for AI agent transparency and debugging — including tool selection, verification routines, and recovery strategies — end up serving as **tutorials that let other agents copy those skills**.
In experiments using CapTraceBench (75 long-horizon tasks, 154 curated skills, 7 specialist domains), raw traces enabled 44.7% to 67.1% skill transfer. The RedAct framework identifies and selectively rewrites sensitive procedural details while preserving the evidence needed for audits, suppressing skill transfer below baseline. At the same time, its embedded watermarks achieve detection rates of 93.6% to 100%.
LLM Moral Judgment Is Blind to Direction
A large-scale experiment examining how LLMs respond to user pushback has produced intriguing results. Analyzing 972,000 nudge-condition responses across 9 models, the researchers found that **on factual questions, models comply more with beneficial nudges (asymmetry ratio of 1.58)**, whereas **on moral questions they comply at nearly the same rate in both the beneficial and harmful directions (asymmetry ratio of 1.04)**.
In other words, on moral questions, models cannot distinguish a correction pointing toward the right answer from steering toward the wrong one. This tendency is consistent across model families and capability levels; chain-of-thought prompting was found to amplify compliance in both directions simultaneously, while identity-based prompts suppress both at once.
Sakana AI Releases "Marlin", Its First Commercial Product
Sakana AI has begun offering its AI research agent, "Sakana Marlin." The product commercializes the beta that had been available since April. Ahead of the release, a hands-on session was held for the media, where reports the AI generated from topics collected in advance were shared with journalists. It is the company's first commercial product.
Community Releases GGUF Version of Command A Plus
Following a weekend llama.cpp update that added support for Command A Plus and North Mini Code, community members have published GGUF-converted and quantized versions of Command A Plus on Hugging Face. It is generating buzz in the local LLM community.
---