SHODAN

AI assisted coding is addictive ...

Patrik Gfeller
Patrik Gfeller

“The Universe is under no obligation to make sense to you.” ― Neil deGrasse Tyson

4 min read

My new local AI is fun, but is at the moment not more than just a toy. Let's see if we can change that.

Project Goal

Project S.H.O.D.A.N. decouples the development workflow from any single IDE or model provider by introducing a pool of specialised agents that own their own inference, exposed via the MCP protocol.

Success criteria

  • Any agent in the pool can be replaced without changing the orchestrator or other agents.
  • The framework runs without VS Code (headless mode) as well as with it.
    Local models (Ollama) and premium APIs (Anthropic, OpenAI) are interchangeable per task.
  • The total cost per task is observable and controllable through model routing rules.

Architecture

Workflow Loop

The workflow loop (Analysis → Planning → Execution → Review with feedback) mirrors proven patterns from LangGraph, AutoGen, and SWE-agent.

Workflow Loop
graph LR
   Task("Task")

   subgraph Flow["Loop"]
      Analysis("Analysis")
      Planning("Planning")
      Execution("Execution")
      Review("Review")
    end

    Done("Done")

    Task --> Analysis
    Analysis  -->|"prompt"| Planning
    Planning  -->|"prompt"| Execution
    Execution -->|"result"| Review
    Review -->|"feedback"| Execution
    Review -->|"result"| Done

  classDef task fill:#f1f8e9,stroke:#5cb85c,stroke-width:2px;
  class Analysis,Planning,Execution,Review task

🧠 Agent Pool

Agent specialization allows matching model capability to task complexity — a small local model for analysis, a premium model for complex reasoning.

I expect MCP to establish itself as the de-facto standard for tool and agent exposure. It offers broad IDE and framework support.

 

Agent Pool
flowchart TD  
  IDE["VS Code"]
  CLI["CLI"]

  IDE --> Orchestrator
  CLI --> Orchestrator

  subgraph Orchestrator["Orchestrator"]
      MCP1["🛠️ Analysis Agent"]
      MCP2["🛠️ Planning Agent"]
      MCP3["🛠️ Execution Agent"]
      MCP4["🛠️ Review Agent"]
  end
  
  Orchestrator -->|"OpenAI API"|LiteLLM
  Orchestrator -->|"MCP"|Github
  LiteLLM["🌐 LiteLLM"]
  Github["🌐 Github"]
   
  classDef agent fill:#e6f7ff,stroke:#0099cc,stroke-width:2px;
  class MCP1,MCP2,MCP3,MCP4 agent;  

Architecture Decisions

DecisionChoiceRationale
Inference proxyLiteLLMOpenAI-compatible, 100+ providers, built-in routing and cost tracking
Agent interfaceMCPEmerging standard, IDE-agnostic, tool-native
OrchestrationPydantic AIMIT licensed, no commercial tier; native Graph/BaseNode model matches the Analysis → Review loop; built-in MCP client; LiteLLM via OpenAI-compatible endpoint; lighter dependency footprint than LangGraph (no LangChain)
Local inferenceOllamaSelf-hosted, supports quantized models, OpenAI-compatible via LiteLLM
Primary languagePythonBest ecosystem for LLM tooling, LangGraph/AutoGen native
Initial Concept

I already marked this idea as deprecated - but after many experiments the concept is worth a follow up after all.

Sandbox

graph TD
    subgraph A["Visual Studio Code"]
        A1["Assistant"] <-.->|"uses"| A2
        subgraph A2["Resources"]
          Agent1("Analysis")

          Agent2("Documentation A")
          Agent3("Documentation B")

          Agent4("Coding A")
          Agent5("Coding B") 
        end

        Agent3 <--> MCP
        Agent5 <--> MCP
        MCP
    end

   MCP <--> |"OpenAPI API"|C

   subgraph C["🌐 Gateway"]
   end

   C <-.->|"Document"| D1
   C <-.->|"Code"| D2

   subgraph D[🧠 Local Inference]
     D1["🛠️ Model A"]
     D2["🛠️ Model B"]
   end

    %% Style Definitions (for visual separation)
    classDef agent fill:#e6f7ff,stroke:#0099cc,stroke-width:2px;
    class A1,A2 agent;
    
    classDef gateway fill:#fffbe6,stroke:#ccaa00,stroke-width:2px;
    class C gateway;
    
    classDef compute fill:#f1f8e9,stroke:#5cb85c,stroke-width:2px;
    class D1,D2 compute;
Working POC ... ready for the next Phase 👍

Phase 0

Phase 1

The Meta-Agent

Links and References:

LlamaIndex and LangChain RAG pipelines

The critical thing to understand before reading any comparison is that these three options are not competing on the same dimension. LangChain is an orchestration toolkit. LlamaIndex is a retrieval toolkit. Raw API calls are a stance on how much abstraction you need. Many production systems use two of them together.

The question is always: given what I am actually building, which layer of abstraction earns its cost?

LLM Orchestration Frameworks Compared

Focus

 

Child pages

Comments

Authors

Recommended Topics