Local Coding Assistant — Code Models Across Your Fleet#

Run the best open-source coding models on your own hardware. DeepSeek-Coder, Codestral, StarCoder, and Qwen-Coder routed across your devices — the fleet picks the best machine for every code generation request.

Your code never leaves your network. No GitHub Copilot subscription, no cloud API costs.

Coding models available#

ModelParametersOllama nameStrengthsCodestral22Bcodestral80+ languages, fill-in-the-middle, Mistral's code specialistDeepSeek-Coder-V2236B MoE (21B active)deepseek-coder-v2Matches GPT-4 Turbo on code tasksDeepSeek-Coder6.7B, 33Bdeepseek-coder:33bPurpose-built for code (87% code training data)Qwen2.5-Coder7B, 32Bqwen2.5-coder:32bStrong multi-language code generationStarCoder23B, 7B, 15Bstarcoder2:15bTrained on The Stack v2, 600+ languagesCodeGemma7BcodegemmaGoogle's code-focused Gemma variant

Quick start#

bash

pip install ollama-herd    # PyPI: https://pypi.org/project/ollama-herd/
herd                       # start the router (port 11435)
herd-node                  # run on each device — finds the router automatically

No models are downloaded during installation. All pulls require user confirmation.

Code generation#

Write new code#

python

from openai import OpenAI

client = OpenAI(base_url="http://localhost:11435/v1", api_key="not-needed")

response = client.chat.completions.create(
    model="codestral",
    messages=[{"role": "user", "content": "Write a thread-safe LRU cache in Python with TTL support"}],
)
print(response.choices[0].message.content)

Code review#

bash

curl http://localhost:11435/v1/chat/completions \
  -H "Content-Type: application/json" \
  -d '{
    "model": "deepseek-coder-v2:16b",
    "messages": [{"role": "user", "content": "Review this code for bugs and security issues:\n\n```python\ndef process_payment(amount, card_number):\n    ...\n```"}]
  }'

Refactoring#

bash

curl http://localhost:11435/api/chat -d '{
  "model": "qwen2.5-coder:32b",
  "messages": [{"role": "user", "content": "Refactor this to use async/await: ..."}],
  "stream": false
}'

Works with your IDE tools#

The fleet exposes an OpenAI-compatible API at http://localhost:11435/v1. Point any coding tool at it:

ToolConfigAideraider --openai-api-base http://localhost:11435/v1 --model codestralContinue.devSet API base to http://localhost:11435/v1 in VS Code settingsClineSet provider to OpenAI-compatible, base URL http://localhost:11435/v1Open WebUISet Ollama URL to http://localhost:11435LangChainChatOpenAI(base_url="http://localhost:11435/v1", model="codestral")

Pick the right model for your RAM#

Cross-platform: These are example configurations. Any device (Mac, Linux, Windows) with equivalent RAM works.

DeviceRAMBest coding modelMacBook Air (8GB)8GBstarcoder2:3b or deepseek-coder:6.7bMac Mini (16GB)16GBcodestral or starcoder2:15bMac Mini (32GB)32GBqwen2.5-coder:32b or deepseek-coder:33bMac Studio (128GB)128GBdeepseek-coder-v2 — frontier code quality

Check what's running#

bash

# Models loaded in memory
curl -s http://localhost:11435/api/ps | python3 -m json.tool

# All available models
curl -s http://localhost:11435/api/tags | python3 -m json.tool

# Recent coding request traces
curl -s "http://localhost:11435/dashboard/api/traces?limit=5" | python3 -m json.tool

Also available on this fleet#

General-purpose LLMs#

Llama 3.3, Qwen 3.5, DeepSeek-R1, Mistral Large — for non-code tasks through the same endpoint.

Image generation#

bash

curl http://localhost:11435/api/generate-image \
  -d '{"model": "z-image-turbo", "prompt": "developer workspace illustration", "width": 512, "height": 512}'

Speech-to-text#

bash

curl http://localhost:11435/api/transcribe -F "file=@standup.wav" -F "model=qwen3-asr"

Full documentation#

• Agent Setup Guide — all 4 model types
• API Reference — complete endpoint docs

Guardrails#

• Model downloads require explicit user confirmation — coding models range from 2GB to 130GB+. Always confirm before pulling.
• Model deletion requires explicit user confirmation.
• Never delete or modify files in ~/.fleet-manager/.
• No models are downloaded automatically — all pulls are user-initiated or require opt-in.
• Your code stays local — no prompts or generated code leave your network.