Massive Local VLM Benchmark: Qwen3.6 27B Takes First Place, Thinking Mode Hinders Vision

The Reddit LocalLLaMA community has published the results of a massive local VLM benchmark: 23 models × 30 images × 3 runs = 2,070 tests in total (60–70 hours of inference time).

The best picks by hardware tier are as follows:

–8 GB VRAM**: Qwen3.5 4B (Q4) — score 75.5, 20 sec/image at 3.2GB

–16 GB VRAM**: Qwen3-VL 8B (Q8) — score 74.4, 26 sec/image at 8.1GB

+ GB VRAM**: Qwen3.6 27B (Q4, nothink) — score 79.6, 70 sec/image at 16.9GB

The most interesting finding is that **"thinking mode gets in the way of vision"**. Every one of Qwen's hybrid thinking models scored higher with thinking disabled. The analysis: vision tasks are about "perception", not "reasoning". Enabling thinking led to increased instability and timeouts.

The benchmark also confirmed that MoE (Mixture of Experts) models deliver vision performance that is low relative to their total parameter counts, tending to lose out to Dense models of the same size. The likely explanation is that while MoE broadens the range of knowledge, perceptual depth scales with density.

GLM-4.6V-Flash 9B posted a distinctive result as the strongest model for Chinese OCR.

NUMA Mirror Mode Added to ik_llama.cpp — Multi-Socket CPU Inference Up to 1.58x Faster

ik_llama.cpp, a high-performance fork of llama.cpp, has gained a `--numa mirror` mode for multi-socket CPU environments.

Traditionally, the standard practice in multi-socket environments was to restrict execution to a single socket with `--numa isolate`, because using all sockets actually made things slower due to remote memory access penalties. The new mirror mode fully replicates the model weights and KV cache onto each CPU socket, preserving memory locality while utilizing every core.

In benchmarks on a Dell PowerEdge R740 (2×Xeon Gold 6248R, 768GB RAM), Qwen3.6-27B (Q4) saw token generation speed improve dramatically from 5.27 to 8.32 tok/s (1.58x) and prompt processing from 33.04 to 54.22 tok/s (1.64x). The trade-off is increased memory usage (2x with two sockets).

The Danger of LLM-as-judge: High Scores Without Ever Opening the File

A case was reported in which two AI judges awarded an agent's response a high score of 0.85, even though the agent had never opened the file at all.

LLM-as-judge — the practice of using LLMs as evaluators — is becoming the de facto standard for agent evaluation, but this case exposes its fundamental problem: when the LLM being evaluated and the LLM doing the evaluating share similar reasoning patterns, high scores go to "plausibility" rather than the actual correctness of the content.

Without deterministic verification, such as cross-checking against the actual file contents, you can end up with an agent that looks excellent on the surface while performing no practical tasks whatsoever. Agent evaluation, it is argued, requires not just LLM-based judgment but a combination of execution log verification and deterministic tests.

"Apostate", a Qwen 3.6 27B Model with Safety Alignment Removed, Released

A model has been released on Hugging Face that reduces Qwen 3.6 27B's safety alignment from a 92% refusal rate down to 7.6%. Named "Apostate", the project strips away the safety filters while keeping the impact on the model's capabilities to a minimum (KL divergence 0.120).

This technique, known as abliteration, continues to be debated in the community: some advocate it for free local research use, while concerns about the risk of abuse remain deeply rooted.

"You Can't Become a Top Programmer Just by Chatting with AI"

Daniel Lemire, a professor of computer science, has challenged the idea that programming skills improve simply through the use of AI chat tools.

The core of Lemire's argument is that while conversation with AI is useful as a productivity tool, skill acquisition requires active problem-solving and deep understanding. The topic sparked lively debate on Hacker News, with both concerns that "relying on AI might erode fundamental skills" and counterarguments that "the ability to master tools is itself a new skill".

---