The AI signal,filtered from the noise.
The agents, models, and labs that actually matter — one sharp email every morning.
Free forever · No hype, no filler · Unsubscribe anytime
Latest from AgentFlow
View all →How LLMs Actually Generate Text: The Full Inference Pipeline
Every token an LLM writes travels the same nine stages: tokenize, embed, run the transformer stack, project to logits, sample, verify, detokenize, stream. The part nobody explains well is that inference splits into two very different phases. Prefill reads your whole prompt in parallel and is compute-bound. Decode writes one token at a time and is memory-bound, gated by a KV cache that grows with every token. Here is the whole pipeline in plain language, why decode is the slow part, and how FlashAttention, quantization, and speculative decoding attack the exact bottleneck that makes it slow.
Claude Sonnet 5: Almost Opus 4.8, at a Third of the Price
Anthropic shipped Claude Sonnet 5 (codename Fennec) on June 30, 2026, and the pitch is simple: near-Opus-4.8 quality at mid-tier prices. It is the most agentic Sonnet yet, ships a 1M-token context window, scores 82.1% on SWE-bench, and lands at $2/$10 per million tokens through August 31. Here is what actually shipped, the real benchmark gap to Opus 4.8, the new tokenizer that quietly changes your bill, the safety gains, and whether you should make it your default driver.
Speculative Decoding, and How DeepSeek DSpark Made Inference Up to 5x Faster
DeepSeek open-sourced DSpark on June 27, 2026, and the headline doing the rounds, "400x faster," is off by a factor. The real number is 50 to 400 percent faster (up to ~5x throughput), 60 to 85 percent lower latency on DeepSeek-V4, with zero retraining and identical output. Here is how speculative decoding actually works, what DSpark adds on top (DFlash, a Markov head, a load-aware scheduler), the honest numbers, and how to use the MIT-licensed DeepSpec toolkit on your own models.
Everything that matters in AI,
straight to your inbox.
Join 12,000+ readers — daily, free, no spam.