RTX 5070 Overtakes RTX 3090 in Local LLM Inference Speed

Reddit user Caleb Coffie has published the results of systematically benchmarking models ranging from 0.35B to 35B parameters across three environments: RTX 5070 (12GB GDDR7), RTX 3090 (24GB GDDR6X), and AMD Strix Halo.

The most notable finding: **for models that fit within 12GB of VRAM, the RTX 5070 beats the RTX 3090 in inference speed**. This reaffirms that GDDR7's high memory bandwidth is the decisive factor in decode performance.

Specific comparisons (short chat):

| Model | RTX 5070 | RTX 3090 |

|---|---|---|

| Gemma-3-4b | 156.6 tok/s | 142.0 tok/s |

| Gemma-4-E4B | 124.3 tok/s | 118.4 tok/s |

| LFM2-8B-A1B | 336.1 tok/s | 318.7 tok/s |

For models in the 14B–31B class, on the other hand, the 3090's 24GB VRAM gives it the edge — with Qwen3.6-27B, the 3090 delivered 21.1 tok/s versus 11.2 tok/s for Strix ROCm, a wide gap.

The impact of quantization was also examined: for Qwen3.6-27B, there was a 1.6x speed difference across the Q2–Q6 range, and Q4_K_M was identified as the sweet spot balancing quality and speed.

The report also notes that Qwen3.5/3.6 reasoning models appear roughly 5x slower than they really are when you look only at output tok/s. Because much of the output flows through a hidden `reasoning_content` channel, there is a large gap between perceived speed and measured numbers.

> **Why it matters**: A consumer GPU (RTX 5070) delivering inference speeds that surpass the previous generation's high-end (RTX 3090) will accelerate the democratization of local LLMs. That said, once model size crosses the 12GB wall, VRAM capacity remains the dominant factor — making this a useful guide for choosing hardware based on your workload.

curl Maintainer: AI Security Reports Are No Longer Slop

Daniel Stenberg, the maintainer of curl, has published a post acknowledging that the quality of AI-generated security reports has improved substantially.

Until now, automatically generated AI security reports — frequently full of false positives and meaningless content — have been dismissed as "slop". Stenberg, however, now assesses that recent AI-generated reports are reaching a practical level.

> **Why it matters**: The flood of AI-generated security reports has been a major burden for OSS projects. If the quality keeps improving, it opens the door to genuinely leveraging AI as a supplementary tool for security audits. Human review remains essential for the time being, though.

AtomicMemory: An Open-Source Memory Engine for AI Agents

A community manager has released AtomicMemory, an open-source engine built for AI agent memory management.

Its standout feature is that users can directly inspect and correct the information an AI has memorized. With conventional AI tools, there has been almost no way to verify or fix incorrectly memorized information — the only option was filing a support ticket.

AtomicMemory consists of five independent domains:

**Ingest** - Classifying and storing memories via AUDN (Add, Update, Delete, Supersede, Clarify, No-op)

**Search** - Semantic-based search (not mere keyword matching)

**CRUD** - Direct memory access without going through an LLM

**Lifecycle** - Memory decay recommendations (deletion stays under user control)

**Trust** - Trust scores assigned on write and referenced on read

It is built on Docker + PostgreSQL + pgvector and supports multiple LLM providers, including Anthropic, OpenAI, Ollama, Google, and Groq.

> **Why it matters**: Memory accuracy and transparency remain unsolved problems for practical AI agent deployment. The risk of an AI continuing to remember incorrect information is especially significant in enterprise settings. An approach that guarantees memory inspection and correction at the architectural level could help make agents more trustworthy.

Sandia National Laboratories Puts AI Visual Inspection into Practical Use

Sandia National Laboratories in the US has announced an AI-powered component inspection system. The effort applies AI image recognition to detecting defects in manufactured parts, aiming to make visual inspection faster and more accurate.

> **Why it matters**: AI adoption in manufacturing keeps advancing, and real-world deployments at the national laboratory level suggest that AI is entering full-scale use in fields with demanding reliability requirements.

Uncensored Fine-Tune of Gemma 4 31B Released

A fine-tuned model based on Gemma 4 31B, "G4-Meromero-31B-Uncensored-Heretic", has been released on Hugging Face. Intended for creative use, it reports metrics of KLD (KL divergence) 0.0100 and a refusal rate of 15/100. It is available in both Safetensors and GGUF formats.

---