Quantifying the LLM "Knowing-Doing Gap," Cutting Reasoning Overthinking by 26%, and a Compute Battle Inside Google

LLMs "Know They Need a Tool Yet Don't Use It" — Quantifying a Discrepancy of Up to 54%

New research shows that in LLM tool use, a "knowing-doing gap" — where a model internally recognizes that a tool is necessary but doesn't actually call it, or conversely calls one it doesn't need — can reach up to 54%.

The paper "Model-Adaptive Tool Necessity Reveals the Knowing-Doing Gap in LLM Tool Use," featured on Hugging Face Daily Papers, introduces the concept of "model-adaptive tool necessity," which defines tool necessity based on each model's own capabilities. Across arithmetic and factual QA tasks, it compares "situations where a tool is actually needed" with "situations where the tool is actually used" across multiple models.

The results are intriguing. Probing the models' hidden states reveals that both the tool-necessity signal and the tool-calling signal are decodable, but in the later layers the two become nearly orthogonal. In other words, "the recognition of whether a tool is needed" and "the action of actually using one" are separated at the level of internal representations.

Furthermore, most of the mismatch was found to be concentrated not in the "recognition" stage but in the "translation from recognition to action" stage. The model understands internally that a tool is needed, yet fails to convert that understanding into an actual tool call. This finding suggests that improving the reliability of agentic AI requires not only more accurate recognition, but also better conversion from recognition to action.

PUMA: Cutting Reasoning Models' "Overthinking" by 26% While Preserving Accuracy

Reasoning-focused LLMs (Large Reasoning Models) achieve high performance through long chains of thought (Chain of Thought), but they often suffer from "overthinking" — continuing to reason even after the answer has already stabilized. This wastes tokens and increases latency.

The newly proposed PUMA (Plug-and-play fraMework with redundancy Analysis) is a framework for detecting and reducing this overthinking. PUMA focuses on "reasoning-level semantic redundancy," detecting when consecutive reasoning steps no longer produce new progress and merely reiterate an already-established conclusion.

The design is simple. A lightweight "redundancy detector" flags semantically redundant candidate exit points, and answer-level verification confirms that stopping is safe. Evaluated on 5 reasoning models and 5 benchmarks, it achieved an average token reduction of 26.2% while maintaining accuracy and the quality of the retained chain of thought.

Transfer experiments on code generation and zero-shot visual-language reasoning also confirmed its effectiveness, showing that reasoning redundancy is a general and learnable signal.

EndPrompt: Extending LLM Context Windows from 8K to 64K with Only Short Sequences

Extending an LLM's context window normally requires training at the target length, incurring quadratic memory and compute costs. A new method called "EndPrompt," however, achieves effective context extension using only short training sequences.

The idea is this: to expose the model to long-range relative position distances, there is no need to construct full-length inputs. Keep the original short context intact as the first segment, and append a short "terminal prompt" as the second segment. This second segment is assigned position indices close to the target context length.

With this two-segment construction, both local and long-range relative distances can be introduced within a short physical sequence. The paper also provides a theoretical analysis based on RoPE (Rotary Position Embedding) and Bernstein's inequality, mathematically showing that position interpolation imposes a smoothness constraint on the attention function.

When applied to extending LLaMA-family models from 8K to 64K, it recorded an average RULER score of 76.03 and the highest average on LongBench — surpassing LCEG (72.24), LongLoRA (72.95), and full-length fine-tuning (69.23) while requiring far less compute. It's a result that overturns the assumption that dense training on long sequences is essential.

Google's Own AI Researchers Fight Over Compute — Bloomberg Reports

According to Bloomberg, competition for compute resources among AI researchers inside Google is intensifying. Google is one of the world's leading AI research institutions and holds vast GPU/TPU resources, but as research projects have proliferated, internal politics over resource allocation have come to the surface.

The problem is not limited to Google — Microsoft, Meta, Amazon, and others are believed to face similar issues. Frontier model training requires thousands to tens of thousands of GPUs, and when a single project monopolizes resources, other projects stall. A situation where research direction hinges on "whether you can secure compute" carries the risk of organizational politics taking priority over science.

This structural problem also matters from the perspective of democratizing AI research. As compute concentrates in a handful of giant corporations, there are growing concerns about the declining competitiveness of academia and smaller labs.

NVIDIA Announces Next-Gen AI Platform "Rubin" and Six New Chips

NVIDIA CEO Jensen Huang has announced a new AI platform, "Rubin," which is already in full production and includes six new chips.

Rubin is positioned as the AI foundation for the next decade, succeeding the current Hopper/Blackwell architectures. Detailed specs remain limited, but it is expected to deliver substantial performance gains in both training and inference to keep up with the explosive growth of AI workloads.

NVIDIA has long maintained an overwhelming share of the AI semiconductor market, but competition is intensifying with AMD and Intel, as well as in-house chips from Google (TPU) and Amazon (Trainium). The Rubin announcement can be seen as a strategic move to maintain and expand that competitive edge.

---

**Sources:**

Model-Adaptive Tool Necessity Reveals the Knowing-Doing Gap in LLM Tool Use — [Hugging Face Papers](https://huggingface.co/papers/2605.14038)

Stop When Reasoning Converges: PUMA — [Hugging Face Papers](https://huggingface.co/papers/2605.17672)

EndPrompt: Efficient Long-Context Extension via Terminal Anchoring — [Hugging Face Papers](https://huggingface.co/papers/2605.14589)

Google's Own AI Researchers Jockey for Access to Its Computing — [Bloomberg](https://www.bloomberg.com/news/articles/2026-05-18/google-s-own-ai-researchers-jockey-for-access-to-its-computing)

NVIDIA Rubin Platform Announcement — [Zenn / AI Quotidia](https://zenn.dev/quotidia/articles/4b40ee42966c82)