SHODAN - Phase 0

Creating the foundation

I did not know Open WebUI when I created the initial plan - but I already have a lot of fun with it.

Open WebUI

Well - inspecting the data sent by visual studio code I decided to change the initial plan and ditch it and try gemma-4 with this.

,,, and it has a plan 😈 ...
docker-compose.yaml
services:
  open-webui:
    image: ghcr.io/open-webui/open-webui:main
    restart: on-failure:3
    container_name: open-webui
    ports:
      - ${PORT_OPEN_WEBUI}:8080
    volumes:
      - ${MOUNT}/app:/app/:rw
    environment:
      - OLLAMA_BASE_URL=${OLLAMA_HOST}
      - ENABLE_AUTOCOMPLETE_GENERATION=False
      - ENABLE_TITLE_GENERATION=False
      - ENABLE_TAGS_GENERATION=False
      - ENABLE_QUERIES_CACHE=True
      - ENABLE_BASE_MODELS_CACHE=True
      - ENABLE_REALTIME_CHAT_SAVE=False
      - DATABASE_ENABLE_SESSION_SHARING=False
      - CHUNK_MIN_SIZE_TARGET=1000
      - RAG_EMBEDDING_ENGINE=ollama
      - RAG_EMBEDDING_MODEL=nomic-embed-text
    dns:
      - ${DNS_1}
  tika:
    image: docker.io/apache/tika:latest
    restart: on-failure:3 
  litellm:
    build:
      context: .
      args:
        target: runtime
    image: docker.litellm.ai/berriai/litellm:main-stable
    volumes:
      - ${MOUNT}/litellm/config/config.yaml:/app/config/config.yaml
    command:
      -  --config=/app/config/config.yaml
    ports:
      - ${PORT_LITELLM_UI}:4000
    environment:
      - OLLAMA_HOST=${OLLAMA_HOST}
      - LITELLM_MASTER_KEY=${LITELLM_MASTER_KEY}
      - LITELLM_SALT_KEY=${LITELLM_SALT_KEY}
      - DATABASE_URL=${LITELLM_POSTGRESQL}
      - STORE_MODEL_IN_DB=True 
    depends_on:
      - db
    healthcheck:
      test:
        - CMD-SHELL
        - python3 -c "import urllib.request; urllib.request.urlopen('http://localhost:4000/health/liveliness')"
      interval: 30s  
      timeout: 10s   
      retries: 3     
      start_period: 40s 
  db:
    image: postgres:16
    restart: on-failure:3
    container_name: litellm_db
    environment:
      POSTGRES_DB: litellm
      POSTGRES_USER: llmproxy
      POSTGRES_PASSWORD: dbpassword9090
    volumes:
      - ${MOUNT_DB}/litellm/postgresql:/var/lib/postgresql/data
    healthcheck:
      test: ["CMD-SHELL", "pg_isready -d litellm -U llmproxy"]
      interval: 1s
      timeout: 5s
      retries: 10
  prometheus:
    image: prom/prometheus
    volumes:
      - prometheus_data:/prometheus
      - ${MOUNT_DB}/litellm/prometheus/prometheus.yml:/etc/prometheus/prometheus.yml
    ports:
      - ${PORT_PROMETHEUS}:9090
    command:
      - "--config.file=/etc/prometheus/prometheus.yml"
      - "--storage.tsdb.path=/prometheus"
      - "--storage.tsdb.retention.time=15d"
    restart: on-failure:3
volumes:
  prometheus_data:
    driver: local
networks:
  proxy:
    name: docker-proxy
    external: true
Boris Karloff als Frankensteins Monster
Boris Karloff als Frankensteins Monster [Wikipedia]

It's alive! Aliiiiive! [🎥]

Status (DOCKHAND)
LiteLLM - Ollama and GitHub Copilot Models

Related

External Links