Deterministic Decision Models

The Foundational Instability of Soft AI Outputs

The prevailing paradigm in LLM application development—relying on models to output descriptive text — is, from a systems engineering perspective, inherently unstable. When an LLM outputs a string — e.g., "billing" — this is not a guarantee; it is a hypothesis. Downstream systems must spend cycles to stabilize the output (regex parsing, string comparison, etc.). This uncertainty exposes the pipeline to linguistic variations.

A dynamic system using models from multiple vendors with different capabilities is vulnerable to such effects.

The Decision Model paradigm, formalized by typesafe.ai, addresses this constraint directly. These models do not generate text for aesthetic consumption; they are specialized, fast systems designed to eliminate ambiguity by returning deterministic, quantifiable outcomes: a choice, a confidence score, and a probability distribution.

Ollama Blog

System One Models in Practice

For S.H.O.D.A.N., which mandates high system integrity, this shift from "guess" to "guaranteed classification" is a very desirable feature.

The Role of Decision Models within the Workflow

The framework is not built around a single model's power; it is built around its capability to create, manage and orchestrate a pool of dynamically created agents.

The Classificaltion Layer is a specialized step to tailor the agentic setup and pipeline to the problem at hand. Decision models have the potential to increase the performance and output quality of this important step significantly.

Jev-style models fit directly as a high-efficiency branch point in the workflow loop, achieving rapid classification before the full resource commitment of multi-agent planning.

Strategic Interface Analysis

The proprietary nature of the endpoint must be viewed through a strategic lens. Standard LLM APIs are designed for broad, general generation. Jev-style models are designed for the narrow, high-speed task of classification.

The MCP protocol mitigates this dependency. It serves as the abstraction layer that abstracts away vendor specifics, ensuring that our agents are interchangeable—whether they execute locally via Ollama or consume a premium cloud API.

Abstraction using MCP

Model Inventory, Licensing, and Cost Model

The Agent Pool utilizes models chosen not just for performance, but for their legal and operational constraints.

The permissive nature of Nimble (Apache 2.0) and Tev1 (MIT) is a non-negotiable success criterion for S.H.O.D.A.N. The system must operate without proprietary legal constraints.

Implementation Status

The integration of this decision model concept was not part of the Phase 1 implementation plan. But a the performance gains are tempting. To evaluate the complexity I'll create a POC.

The project already uses Pydantic AI for type-safe orchestration. This allows us to utilize the inherent type-safety of Pydantic while leveraging the high-speed, deterministic output of Jev models without the need for manual, brittle API wrappers.

Pydantic AI Documentation

💻 Concept 

The mechanism involves defining the expected schema (the questions) and feeding it into the state (the ticket). The following snippet demonstrates the required function call signature.

from typesafe_sdk import Choice, Noul, Score, TypeSafeClient

customer_ticket = "I was charged twice. Please refund the extra payment."
decision_questions = {
    "team": Choice(
        instructions="Which team should handle this ticket?",
        criteria={"billing": "Payments and refunds", "technical": "Bugs and integrations", "other": "None of the above"},
    ),
    "refund": Noul(
        instructions="Does the customer explicitly ask for a refund?",
    ),
    "urgency": Score(
        instructions="How urgent is this ticket?",
        criteria=["Routine", "Soon", "Urgent"],
    ),
}

try:
    with TypeSafeClient(timeout=120) as client:
        result = client.system_one(
            state={"ticket": customer_ticket},
            questions=decision_questions,
        )
    # Results are now structured and ready for downstream routing logic.
except Exception as e:
    # This is the point where Phase 2 work prevents a simple script error.
    pass

Conclusion

The Jev-style models are not a "quick win"; but using them as high-performance gate(s) in the workflow will free resources for the more heavyweight LLM reasoning tasks, that genuinely require creative capacity.

By focusing on this kind of constrained, deterministic classification, we might be able to improve the ROI of the available computational resources and minimizes the risk profile of the operational layer.

This arcticle and the illustrations were created using AI tools and agents.