SHODAN

AI assisted coding is addictive ...

Patrik Gfeller

My new local AI is fun, but is at the moment not more than just a toy. Let's see if we can change that.

Project Goal

Project S.H.O.D.A.N. decouples the development workflow from any single IDE or model provider by introducing a pool of specialised agents that own their own inference, exposed via the MCP protocol.

Success criteria

  • Any agent in the pool can be replaced without changing the orchestrator or other agents.
  • The framework runs without VS Code (headless mode) as well as with it.
    Local models (Ollama) and premium APIs (Anthropic, OpenAI) are interchangeable per task.
  • The total cost per task is observable and controllable through model routing rules.

Architecture

Workflow Loop

The workflow loop (Analysis → Planning → Execution → Review with feedback) mirrors proven patterns from LangGraph, AutoGen, and SWE-agent.

Workflow Loop
graph LR
   Task("Task")

   subgraph Flow["Loop"]
      Analysis("Analysis")
      Planning("Planning")
      Execution("Execution")
      Review("Review")
    end

    Done("Done")

    Task --> Analysis
    Analysis  -->|"prompt"| Planning
    Planning  -->|"prompt"| Execution
    Execution -->|"result"| Review
    Review -->|"feedback"| Execution
    Review -->|"result"| Done

  classDef task fill:#f1f8e9,stroke:#5cb85c,stroke-width:2px;
  class Analysis,Planning,Execution,Review task

🧠 Agent Pool

Agent specialization allows matching model capability to task complexity — a small local model for analysis, a premium model for complex reasoning.

I expect MCP to establish itself as the de-facto standard for tool and agent exposure. It offers broad IDE and framework support.

 

Agent Pool
flowchart TD  
  IDE["VS Code"]
  CLI["CLI"]

  IDE --> Orchestrator
  CLI --> Orchestrator

  subgraph Orchestrator["Orchestrator"]
      MCP1["🛠️ Analysis Agent"]
      MCP2["🛠️ Planning Agent"]
      MCP3["🛠️ Execution Agent"]
      MCP4["🛠️ Review Agent"]
  end
  
  Orchestrator -->|"OpenAI API"|LiteLLM
  Orchestrator -->|"MCP"|Github
  LiteLLM["🌐 LiteLLM"]
  Github["🌐 Github"]
   
  classDef agent fill:#e6f7ff,stroke:#0099cc,stroke-width:2px;
  class MCP1,MCP2,MCP3,MCP4 agent;  

Architecture Decisions

DecisionChoiceRationale
Inference proxyLiteLLMOpenAI-compatible, 100+ providers, built-in routing and cost tracking
Agent interfaceMCPEmerging standard, IDE-agnostic, tool-native
OrchestrationPydantic AIMIT licensed, no commercial tier; native Graph/BaseNode model matches the Analysis → Review loop; built-in MCP client; LiteLLM via OpenAI-compatible endpoint; lighter dependency footprint than LangGraph (no LangChain)
Local inferenceOllamaSelf-hosted, supports quantized models, OpenAI-compatible via LiteLLM
Primary languagePythonBest ecosystem for LLM tooling, LangGraph/AutoGen native
Initial Project Plan (deprecated)
    graph TD
    subgraph A["Visual Studio Code"]
        A1["Assistant"] <-.->|"uses"| A2
        subgraph A2["Resources"]
          Agent1("Analysis")

          Agent2("Documentation A")
          Agent3("Documentation B")

          Agent4("Coding A")
          Agent5("Coding B") 
        end

        Agent3 <--> MCP
        Agent5 <--> MCP
        MCP
    end

   MCP <--> |"OpenAPI API"|C

   subgraph C["🌐 Gateway"]
   end

   C <-.->|"Document"| D1
   C <-.->|"Code"| D2

   subgraph D[🧠 Local Inference]
     D1["🛠️ Model A"]
     D2["🛠️ Model B"]
   end

    %% Style Definitions (for visual separation)
    classDef agent fill:#e6f7ff,stroke:#0099cc,stroke-width:2px;
    class A1,A2 agent;
    
    classDef gateway fill:#fffbe6,stroke:#ccaa00,stroke-width:2px;
    class C gateway;
    
    classDef compute fill:#f1f8e9,stroke:#5cb85c,stroke-width:2px;
    class D1,D2 compute;
  
gantt
    title       Project Plan
    dateFormat  YYYY-MM-DD
    axisFormat  Wk %W
    tickInterval 1week

    section Phase 0 — Foundation
    Framework evaluation                 :p0a, 2000-01-03, 7d
    LiteLLM + Ollama setup               :p0b, 2000-01-03, 7d
    MCP file server proof of concept     :p0c, after p0a, 7d
    VS Code MCP registration test        :p0d, 2000-01-15, 3d
    Phase 0                              :milestone, m0,after p0c, 0d

    section Phase 1 — MVP
    Orchestrator core                    :p1a, after m0,  14d
    Analysis agent (local model)         :p1b, after p1a, 7d
    Execution agent (premium model)      :p1c, after p1a, 7d
    GitHub MCP tools                     :p1d, after p1b, 7d
    CLI entrypoint                       :p1e, after p1c, 3d
    Phase 1                              :milestone, m1,  after p1c, 0d

    section Phase 2 — Pool Expansion
    Review agent + feedback loop         :p2a, after m1,  10d
    Planning agent                       :p2b, after p2a, 7d
    Cost ceiling enforcement             :p2c, after p2a, 5d
    Structured logging                   :p2d, after p2c, 3d
    Phase 2                              :milestone, m2,  after p2c, 0d

    section Phase 3 — VS Code Decoupling
    Headless mode — Linux                :p3a, after m2,  7d
    Agent hot-swap                       :p3b, after m2,  10d
    Integration testing                  :p3c, after p3a, 5d
    Phase 3                              :milestone, m3,  after p3c, 0d

Links and References:

LlamaIndex and LangChain RAG pipelines

The critical thing to understand before reading any comparison is that these three options are not competing on the same dimension. LangChain is an orchestration toolkit. LlamaIndex is a retrieval toolkit. Raw API calls are a stance on how much abstraction you need. Many production systems use two of them together.

The question is always: given what I am actually building, which layer of abstraction earns its cost?

LLM Orchestration Frameworks Compared

Tools:

Child pages

Authors

Recommended Topics