"Geometry Conflict" Is the Real Culprit Behind Forgetting in Continual LLM Training

Research featured on Hugging Face Daily Papers formulates the cause of catastrophic forgetting in continual post-training of LLMs as "geometry conflict" rather than large parameter updates.

The research team argues that forgetting is not simply a matter of updates being too large, but a "state-relative failure of update integration": updates for new tasks become geometrically incompatible with the model's current state as shaped by past updates. They introduce "geometry conflict", a signal grounded in covariance geometry, and compare it against update norms, subspace alignment, and gradient conflict on Qwen3 0.6B-14B, showing that state-relative geometry conflict best captures when transfer and interference occur.

Building on this insight, they propose "Geometry-Conflict Wasserstein Merging" (GCWM), a data-free method for integrating continual updates. Using geometry conflict as a gate for Wasserstein-based merge corrections, it improves retention and final performance without any replay data. The implication — that continual learning should control not just how much the model moves but whether each update stays geometrically compatible with the evolving model state — looks set to shape how LLMs are operated going forward.

Optimizing ubatch in llama.cpp — Prompt Processing at 5.5x Speed

A post on Reddit's LocalLLaMA community reported, with measured numbers, that optimizing llama.cpp's `-ub` (ubatch) parameter dramatically speeds up prompt processing.

In a test running gpt-oss-120b-F16.gguf on an RTX 3090 (24GB VRAM), prompt processing that reached 380.27 tok/s with the default `-ub 512` climbed to 1682.47 tok/s with `-ub 4096` and 2090.68 tok/s with `-ub 8192` — roughly a 5.5x speedup over the default. Token generation, meanwhile, dipped only about 7%, from 32.29 tok/s to 30.05 tok/s.

The trade-off: larger ubatches need more GPU compute workspace, so some MoE layers have to be offloaded to the CPU (bump up `--n-cpu-moe`). As practical tuning wisdom — dramatic gains for prompt-heavy workloads with only a minor hit to generation speed — it's well worth a look.

"Entity Identity Confusion" Lurking in Multimodal Knowledge Editing

New research identifies "Entity Identity Confusion" (EIC), a systemic failure mode that emerges after knowledge editing in large vision-language models (VLMs).

EIC shows up as puzzling behavior: in an edited model, text-only queries about the original entity's identity return information about the new entity. The research team built a diagnostic benchmark, EC-Bench, and pinned down the root cause — existing methods fail to distinguish image-entity binding (I-E) from inter-entity relational knowledge (E-E).

In effect, the image is still recognized as the original entity, with the new entity's name serving as little more than a fake label. The work also shows that constraining edits to the image-entity processing stage substantially reduces EIC, offering guidance toward faithful multimodal knowledge editing.

Auto-Rubric as Reward — Auto-Generating VLM Evaluation Rubrics from a Few Labels

A method called "Auto-Rubric as Reward" has been announced that automatically generates human-readable rubric text from a small set of labeled visual data and puts it to work in vision-language model (VLM) evaluation.

The method supports VLM grading in both pointwise and pairwise settings, letting practitioners freely scale up the rubric dimensions they care about. It also provides a concise pairwise online RL algorithm for diffusion models, designed with an emphasis on data efficiency, training stability, and scalability.

Its application to multimodal generation tasks such as text-to-image generation and image editing has also been validated, bringing a fresh approach to generative AI quality control through automated reward design.

Cisco CPO: "AI Will Build the Majority of Our Products by the End of 2027"

Cisco's Chief Product Officer has predicted that by the end of 2027, AI will be building the majority of the company's products. According to SDxCentral, the sight of a major networking vendor's technology leader expressing such confidence in the rapid advance of AI-driven product development is drawing attention as a remark symbolic of how AI adoption is accelerating across the tech industry.

Beyond Cisco, AI's role in software development is expanding fast, with adoption spreading beyond coding into design, testing, and operations. That said, what exactly counts as AI "building" a product, and where human review fits in, will likely be worked out through practice in the years ahead.

Wrap-Up

This digest covered research that takes a geometric angle on the fundamental challenges of continual LLM training, immediately useful llama.cpp tuning findings from the local LLM community, and an analysis of unexpected side effects in VLM knowledge editing. All three share a practical focus on how to work with existing models, speaking directly to the challenges teams face in production.