Introduction: A Day Dominated by Moves to Reshape the Inference Landscape
Looking back at the AI news collected on July 25, one theme stands out: how to make LLM inference more efficient. While models keep growing larger, we saw a wave of announcements related to running bigger models in resource-constrained environments, through innovations in hardware, architecture, and execution environments.
Today we're covering six topics from that batch. As for the runaway OpenAI agent incident we covered last time, additional information was limited this round, so we won't be revisiting it here.
1. Fujitsu's New Optical AI Architecture "PHOTON" Claims a "475x" Improvement
Fujitsu's newly proposed AI architecture, "PHOTON," is drawing attention. According to an article by Dev_haru on Qiita, who published a technical deep-dive, PHOTON applies optical communication technology to the communication paths of AI processing, claiming up to a 475x efficiency improvement over conventional approaches.
The article works through the numbers touted in the press release, examining the outline of the approach and the basis for the figures in detail. The approach appears to center on reducing inter-chip communication latency with optical technology and avoiding memory bandwidth bottlenecks, but since we haven't verified the original paper or related sources for this piece, please refer to that explainer article and Fujitsu's official announcement for details.
2. Slipstream: Streaming MoE Experts from SSD, Running 35B–480B Models on a 36GB Mac
Drawing attention on Reddit's r/LocalLLaMA is "Slipstream." According to its developer, Illustrious-Cup-5895, it's a fork of llama.cpp that implements a mechanism for streaming the expert weights of MoE (Mixture of Experts) models from an SSD on demand.
In MoE, only a small subset of experts fires per token, which leaves most weights idle. Slipstream keeps only the weights actually needed resident in RAM, streams routed experts in as they're needed, and manages them with a constrained cache to prevent swapping. The upshot, the developer explains, is that quantized models ranging from 35B to 480B can run even on a 36GB MacBook.
What's particularly interesting is that it's being published "failures included." The post reports real-world numbers of roughly 13 tok/s on Qwen3.6-35B-A3B (Q4) with a 10GiB cache and about 2.8 tok/s on Laguna 118B-A8B — while also candidly reporting that dual-SSD striping and speculative prefetching both ended in negative results. It's available as an MIT-licensed, fully offline native macOS app, and was optimized for Apple Silicon with reference to Colibri.
3. TensorSharp: An LLM Inference Engine Written in C#, Claiming Performance on Par with llama.cpp
Also posted on r/LocalLLaMA, by fuzhongkai, "TensorSharp" is an LLM inference engine implemented from scratch in C#. It's not a C# wrapper around llama.cpp — the entire LLM inference engine was built bottom-up.
Comparative benchmarks against llama.cpp on the CUDA and Vulkan backends have been published for Gemma 4 E4B / 12B and Qwen 3.6 35B-A3B / 27B. It surpassed llama.cpp in some scenarios: 1.00–1.21x on _decode_ (token generation), up to 1.28x on _prefill_, and up to 1.27x on TTFT. That said, results vary by backend and model — Qwen 3.6 27B on Vulkan came in at 0.84–1.02x.
It touts multi-platform support across Windows/macOS/Linux for Nvidia, Apple, AMD, Intel, and other hardware, with an OpenAI/Ollama-compatible API. It also incorporates advanced techniques such as paged KV caching and continuous batching derived from vLLM, and an SSD-based cache for MoE derived from oMLX.
4. LFM 2.5 230M: 1440 tok/s in the Browser via WebGPU
Efforts to run LLMs in the browser via WebGPU are also moving forward. In a demo released by lordhiggsboson, LFM 2.5 230M reportedly hit 1400–1500 tok/s on an RTX 3090 setup and 400–500 tok/s on an Apple M4. Bonsai 1.7B also measured 500–600 tok/s on the RTX 3090.
What deserves attention is the hardware-specific design of the optimizations. For Nvidia, kernels are aggressively fused within a multi-pass structure, while on Apple Silicon it adopts a single fused megakernel to minimize the overhead of TBDR (Tile Based Deferred Rendering). The actual demo is live at https://warp.sipp.sh, and the code is slated to be integrated into the Sipp library going forward.
5. Pyshackle: An OSS That Puts a "Pre-Execution Gate" on AI Agent Tool Calls
As AI agents become more autonomous, demand is growing for a security layer that prevents unintended tool executions. "Pyshackle," published on PyPI, provides a mechanism that inserts a "hard pre-execution gate" before an agent calls a tool. It's open source and has also been picked up on Hacker News.
Since implementation details are limited in the data we collected for this piece, please refer to the original (pypi.org/project/pyshackle/) for configurable gate conditions and supported runtimes. Viewed alongside Slipstream's "publish your failures too" stance, it's a sign that designing AI agent safety is becoming a major engineering theme.
6. AlayaWorld: An Interactive, Long-Horizon 15B Video World Model Arrives as Open Source
Featured on Papers with Code, "AlayaWorld" is an interactive video world model capable of long-horizon generation. Built on a 15B video diffusion transformer, it generates 24fps video at 540p/720p. It works by autoregressively generating short latent chunks, following camera trajectories and switchable text prompts.
Particularly clever is the "bounded visual context" design. It combines persistent sink frames, compressed temporal history, geometrically consistent spatial memory, and conditioning on the most recent frames to reduce drift during long-horizon generation. It also employs an approach that leverages corrupted histories and prediction residuals from self-rollouts during training, and "discrete autoregressive distillation" — combining distribution matching distillation, self-forcing++, and consistency distillation — cuts inference from roughly 30 steps per chunk down to 4. The authors claim top performance on iWorld-Bench for long-horizon generation. It's being released as a full-stack, open-source long-term project.
Editor's Note: Grounded, Incremental Ingenuity in LLM Inference Keeps Accelerating
Looking at the news from the latter half of this week, infrastructure-flavored work on "how to run" existing models stands out far more than "training" ever-larger ones. From injecting optical technology (Fujitsu's PHOTON), to SSD streaming (Slipstream), reimplementation in a managed-language runtime (TensorSharp), direct access to GPU resources (WebGPU browser LLMs), a safety valve for agent execution (Pyshackle), and longer, fewer-step video generation (AlayaWorld) — the layers differ, but a common concern runs through them all: how to squeeze the most out of limited resources.
Please refer to the original sources for detailed figures and for verifying the implementations. This article stays within the bounds of the data we collected and deliberately avoids overconfident claims.