Why Enterprises Have Started Putting AI Usage on 'Rationing'

According to a report by The Wall Street Journal, 'rationing' of AI usage has begun among U.S. companies. To cope with soaring AI costs, companies are being forced to cap how much AI each employee can use and to prioritize resource allocation for specific departments.

Backing up this reporting, an analysis from CodeScene offers an interesting insight: on projects with low-quality ('unhealthy') code, **AI agents consume 35–50% more tokens**. The structure is such that as the AI repeatedly struggles to understand complex, hard-to-read code, inference costs balloon. For enterprises pushing AI adoption, this presents a new perspective: paying down technical debt is no longer just a developer-productivity issue but translates directly into cost savings.

Monokernel Inference Reaches 3,300 tokens/s on AMD MI300X

A developer team that drew attention on Reddit's r/MachineLearning has built a 'monokernel' that runs the full decode sequence of LLM inference as a single GPU-resident program on AMD MI300X.

What makes it distinctive is the optimization that takes advantage of the MI300X's physical die topology. By mapping memory access patterns to the physical layout and grouping compute units by their associated IODs, the team squeezes out the hardware's full designed performance. On 8 MI300X GPUs at batch size 1, with no speculative decoding and no quantization, they achieved **up to 3,300 tokens/s**.

Currently it is a preview running on a small 2B coding model, but support for frontier MoE models is planned as well. In AI infrastructure aiming to break away from NVIDIA dependence, AMD's presence is growing.

llama.cpp Saves Flash Attention VRAM with f16 Masks

A welcome improvement for the local LLM community has landed in llama.cpp. A PR that keeps the masks for Flash Attention (FA) in f16 precision has been merged, reducing VRAM consumption compared to before.

It's a practical improvement that makes it possible to handle larger models and longer context lengths on GPUs with limited VRAM, such as the 3090. Voices from the community are already celebrating: 'Now I can download big models again.'

AI Agent Permission Management Is Still Being Explored Between 'Works' and 'Safe'

A ScaleX blog post maps out the challenges of permission management for AI agents. There is a big gap between the 'working' state and the 'safe' state, and how to design this missing layer looks set to become the key to making agentic AI practical.

FreeBSD Undergoes an AI Code Audit

An AI-based audit was conducted on the FreeBSD source code, and the results have been published on a blog. As a case study of applying AI to OSS projects, it demonstrates the potential to detect security vulnerabilities through an approach distinct from static analysis tools.

---