New Optimizer 'STAM' Arrives as an Alternative to AdamW's Fixed Momentum
TokenAI has announced a new optimization algorithm named "Stable Training with Adaptive Momentum" (STAM). Optimizers such as Adam, AdamW, and LAMB have long been the standard for training AI models, but they all share a limitation: a fixed beta1 (momentum coefficient).
STAM's core idea is to measure the difference between the current gradient and past momentum, dynamically lowering beta1 when the gap is large — a mechanism that maintains stability even during noisy training phases. It delivers adaptive behavior: momentum stays high while gradients are stable and is automatically dampened as noise increases.
The lightweight variant, STAMLite, is designed as a drop-in replacement for AdamW and requires only about half the optimizer state memory. While AdamW needs roughly 2x the parameter size in memory, STAMLite needs only about 1x, effectively cutting GPU usage by 50%.
In benchmarks, it achieved an accuracy of 0.61 and a loss of 0.91 in hyperparameter sweeps, and matched NAdam for the top spot on long-horizon non-stationary MLP tasks with an accuracy of 0.97 and a loss of 0.09.
Efforts to train LLMs from scratch with STAM have already begun, and further validation of its practicality is expected.
Italian PM Meloni Shares an AI-Generated Deepfake of Herself
Italian Prime Minister Giorgia Meloni posted to her own social media a photo, apparently synthesized with AI, showing her in underwear, warning of the danger of fabricated false images.
A politician publicly sharing a deepfake image of herself to raise the issue symbolizes just how serious an impact AI image generation technology is having on society. As the quality of AI-generated images improves, the risk of fake images spreading is growing not only for political targets but also for ordinary citizens, once again underscoring the importance of countermeasure technologies.
Critical Debate Over the Direction of Mechanistic Interpretability Research
A lively debate about the current state of mechanistic interpretability research is underway in the Reddit machine learning community.
The field drew attention around 2024 with techniques such as sparse autoencoders (SAEs) and attribution graphs, but skeptical voices have recently been raised about Anthropic's "Natural Language Autoencoders" (NLA). NLA compresses activations from one LLM into natural language descriptions and reconstructs them with another LLM, but critics point out that it is a black-box approach and lacks basic comparison metrics against SAE baselines (such as FVE and reconstruction error).
There is also criticism of a "confabulation" problem, in which NLA's activation-verbalization module generates explanations of activations that may be fabricated. If it is impossible to determine at test time whether an explanation is confabulated, the approach fails the fundamental purpose of interpretability, critics argue.
At the root of the debate lies concern that mechanistic interpretability is shifting away from its original goal of "understanding model internals" toward becoming a tool for scalable alignment and monitoring.
AI Agent Monitoring Log Tool 'AgentBlackBox' Released
A Python package called AgentBlackBox has been released. It is a "flight recorder" for AI agent activity: adding a single line of decorators lets you record every LLM call, tool execution, error, and cost.
It can be installed with `pip install agentblackbox`. With demand rising for retroactively tracing what happened during agent debugging and operational monitoring, this simple setup is a practical addition.
---