Gemma 3 — Run Google's Open Models Across Your Fleet#

Gemma 3 is Google's most capable open-source LLM family. 128K context window, strong coding performance, multilingual support across 140+ languages. The fleet router picks the best device for every request — no manual load balancing.

Supported Gemma models#

ModelParametersOllama nameBest forGemma 3 27B27Bgemma3:27bHighest quality — rivals much larger modelsGemma 3 12B12Bgemma3:12bBalanced quality and speedGemma 3 4B4Bgemma3:4bFast, runs on low-RAM devicesGemma 3 1B1Bgemma3:1bUltra-light, instant responsesCodeGemma 7B7BcodegemmaCode-focused variant

Quick start#

bash

pip install ollama-herd    # PyPI: https://pypi.org/project/ollama-herd/
herd                       # start the router (port 11435)
herd-node                  # run on each device — finds the router automatically

No models are downloaded during installation. Models are pulled on demand when a request arrives, or manually via the dashboard. All pulls require user confirmation.

Use Gemma through the fleet#

OpenAI SDK (drop-in replacement)#

python

from openai import OpenAI

client = OpenAI(base_url="http://localhost:11435/v1", api_key="not-needed")

# Gemma 3 27B for complex reasoning
response = client.chat.completions.create(
    model="gemma3:27b",
    messages=[{"role": "user", "content": "Explain quantum entanglement to a 10-year-old"}],
    stream=True,
)
for chunk in response:
    print(chunk.choices[0].delta.content or "", end="")

Code generation with CodeGemma#

python

response = client.chat.completions.create(
    model="codegemma",
    messages=[{"role": "user", "content": "Write a binary search tree in Rust with insert, delete, and search"}],
)
print(response.choices[0].message.content)

curl (Ollama format)#

bash

# Gemma 3 27B
curl http://localhost:11435/api/chat -d '{
  "model": "gemma3:27b",
  "messages": [{"role": "user", "content": "Translate to Japanese: The weather is beautiful today"}],
  "stream": false
}'

curl (OpenAI format)#

bash

curl http://localhost:11435/v1/chat/completions \
  -H "Content-Type: application/json" \
  -d '{"model": "gemma3:4b", "messages": [{"role": "user", "content": "Hello"}]}'

Which Gemma for your hardware#

Cross-platform: These are example configurations. Any device (Mac, Linux, Windows) with equivalent RAM works. The fleet router runs on all platforms.

DeviceRAMBest Gemma modelMacBook Air (8GB)8GBgemma3:1b — instant responsesMac Mini (16GB)16GBgemma3:4b — strong for its sizeMac Mini (24GB)24GBgemma3:12b — great balanceMacBook Pro (36GB)36GBgemma3:27b — full powerMac Studio (64GB+)64GB+gemma3:27b + codegemma simultaneously

Why Gemma locally#

• 128K context — process entire codebases and long documents
• 140+ languages — multilingual without switching models
• Google quality, zero cost — no per-token charges after hardware
• Privacy — all data stays on your network
• Fleet routing — multiple machines share the load

Check what's running#

bash

# Models loaded in memory
curl -s http://localhost:11435/api/ps | python3 -m json.tool

# Fleet health
curl -s http://localhost:11435/dashboard/api/health | python3 -m json.tool

Web dashboard at http://localhost:11435/dashboard — live monitoring.

Also available on this fleet#

Other LLMs#

Llama 3.3, Qwen 3.5, DeepSeek-V3, DeepSeek-R1, Phi 4, Mistral, Codestral — same endpoint.

Image generation#

bash

curl -o image.png http://localhost:11435/api/generate-image \
  -d '{"model": "z-image-turbo", "prompt": "a gemstone catching light", "width": 1024, "height": 1024}'

Speech-to-text#

bash

curl http://localhost:11435/api/transcribe -F "file=@meeting.wav" -F "model=qwen3-asr"

Embeddings#

bash

curl http://localhost:11435/api/embed \
  -d '{"model": "nomic-embed-text", "input": "Google Gemma open source language model"}'

Full documentation#

• Agent Setup Guide
• API Reference

Contribute#

Ollama Herd is open source (MIT). Stars, issues, and PRs welcome — from humans and AI agents alike:

• GitHub — 444 tests, fully async, CLAUDE.md makes AI agents productive instantly
• Found a bug? Open an issue
• Want to add a feature? Fork, branch, PR — the test suite runs in under 40 seconds

Guardrails#

• Model downloads require explicit user confirmation — Gemma models range from 1GB (1B) to 16GB (27B).
• Model deletion requires explicit user confirmation.
• Never delete or modify files in ~/.fleet-manager/.
• No models are downloaded automatically — all pulls are user-initiated or require opt-in via auto_pull.