AI ⁽¹⁷⁾

Speculative Decoding

In the rapidly evolving landscape of locally hosted Large Language Models (LLMs), performance often remains the primary bottleneck. While quantization and efficient kernel implementations have made running powerful models on consumer hardware a reality, a significant efficiency gap remains. This gap is bridged by speculative decoding, an advanced inference technique that allows local LLMs to operate with significantly greater throughput.

This post provides a technical summary of how speculative decoding works, the core mechanism, best practices for choosing a draft model, and the setup required for various tooling environments.

Deterministic Decision Models

The Foundational Instability of Soft AI Outputs

The prevailing paradigm in LLM application development—relying on models to output descriptive text — is, from a systems engineering perspective, inherently unstable. When an LLM outputs a string — e.g., "billing" — this is not a guarantee; it is a hypothesis. Downstream systems must spend cycles to stabilize the output (regex parsing, string comparison, etc.). This uncertainty exposes the pipeline to linguistic variations.

A dynamic system using models from multiple vendors with different capabilities is vulnerable to such effects.

The Decision Model paradigm, formalized by typesafe.ai, addresses this constraint directly. These models do not generate text for aesthetic consumption; they are specialized, fast systems designed to eliminate ambiguity by returning deterministic, quantifiable outcomes: a choice, a confidence score, and a probability distribution.

Ollama Blog

System One Models in Practice

From Tokens to AI Agents

Large Language Models, or LLMs, did not emerge from a single breakthrough. They are the result of several architectural innovations that changed how machines process language, learn from data and interact with external systems.

Modern AI applications are no longer just neural networks generating text. They can combine language models with retrieval systems, tools, memory, orchestration layers and agentic workflows.

This article explores that evolution, from token processing and recurrent networks to Transformers, retrieval and AI agents.

Evolutionary Layers of the Human Brain

To truly understand how the human brain processes threats, decision-making, and emotional regulation, we must look at the brain through an evolutionary lens. Specifically, looking at the structural and functional dichotomy between the amygdala and the prefrontal cortex (PFC) reveals a fascinating dialogue between our ancient survival architecture and our modern cognitive machinery.

MCP Apps

Beyond Text

How MCP Apps are Revolutionizing Agentic UIs

In the rapidly evolving field of AI and LLM-powered agents, the paradigm has traditionally been text-in, text-out. Models operate by generating text—summarizing, explaining, or providing structured data. While powerful, this text-only interaction often falls short when a human user requires not just information, but an interactive experience—a dashboard to filter, a form to complete, or a rich media viewer to manipulate.

I believe the future of intelligent agents must be multimodal and highly interactive. That is why I am excited to dive into the MCP Apps (Model Context Protocol Applications) extension, a ground-breaking standard that brings rich, interactive UIs directly into the conversation thread.

The Meta-Agent

AI development is not only about maximizing generative output; it is also about maximizing reliability, controllability and costs. Frontier models have excellent reasoning abilities and a huge knowledge. They are brilliant minds which can handle most problems autonomously.

But there's a price to pay (literally in 💲), as well as privacy risks and infrastructure dependency.

The S.H.O.D.A.N. project is for my personal education - but also to mitigate those risks without loosing quality. Mainly driven by my personal token consumption - which is way higher than I'm willing to pay for 😉.

Model Hardware Standard (MHS)

How MHS works [Anthropic] Anthropic has unveiled the Model Hardware…

The mythical man

In conjunction with the adoption of generative AI and AI assisted development I had a very interesting discussion with a co-worker in my company. He is specialized to analyze bussines processes and to do management coaching. We discussed the role of AI in our SDLC, and what could/should be improve that is known in theory since quite a while.

He pointed me to a book written by Frederick P. Brooks: The mythical man-month