Private AI — Your Data Never Leaves Your Network#

Private AI means every prompt, every response, every image, every transcription stays on your machines. Private AI requires no cloud APIs. Private AI sends no telemetry. Private AI has no third-party access. This is offline AI that runs entirely on your own hardware.

What makes this private AI#

• Private AI networking — the router and nodes communicate only on your local network
• Private AI telemetry — zero. No usage data, no analytics sent anywhere
• Private AI credentials — none needed. No API keys, no accounts, no tokens
• Offline AI capable — models run fully offline after download. Air-gapped AI deployment supported
• Private AI data storage — all data in ~/.fleet-manager/ on your machines only
• Air-gapped AI — pre-download models, then disconnect. The private AI fleet runs without internet

Setup private AI#

bash

pip install ollama-herd    # install private AI router
herd                       # start private AI router (port 11435)
herd-node                  # run on each device — private AI nodes discover each other

No models downloaded during installation. All private AI model downloads require explicit user confirmation. Once downloaded, your private AI runs entirely offline.

Private AI for LLM inference#

Send sensitive prompts with private AI — legal documents, medical notes, financial data, proprietary code never leave your network.

python

from openai import OpenAI

# Private AI endpoint — all inference stays local
private_ai = OpenAI(base_url="http://localhost:11435/v1", api_key="not-needed")

# Private AI document analysis — data never leaves your network
response = private_ai.chat.completions.create(
    model="llama3.3:70b",  # private AI inference on your hardware
    messages=[{"role": "user", "content": "Review this confidential contract for risks using private AI"}],
)
print(response.choices[0].message.content)

Private AI with any model — all fully offline#

bash

# Private AI with DeepSeek-R1 — no data sent to cloud
curl http://localhost:11435/api/chat -d '{
  "model": "deepseek-r1:70b",
  "messages": [{"role": "user", "content": "Analyze this private financial report with offline AI"}],
  "stream": false
}'

Private AI image generation#

Generate images from sensitive prompts with private AI — no DALL-E, no Midjourney, no cloud upload.

bash

# Private AI image generation — prompt stays local
curl http://localhost:11435/api/generate-image \
  -d '{"model": "z-image-turbo", "prompt": "confidential product mockup generated by private AI", "width": 1024, "height": 1024}'

Private AI transcription#

Private AI transcription for meetings, legal depositions, medical dictation — no cloud STT.

bash

# Private AI speech-to-text — audio never leaves your network
curl http://localhost:11435/api/transcribe \
  -F "file=@private_board_meeting.wav" \
  -F "model=qwen3-asr"

Private AI embeddings#

Build private AI knowledge bases from proprietary documents — no OpenAI embedding API.

bash

# Private AI embeddings — document content stays local
curl http://localhost:11435/api/embed \
  -d '{"model": "nomic-embed-text", "input": "Q4 revenue projections — private AI embedding for internal RAG"}'

Air-gapped AI / offline AI deployment#

For fully disconnected private AI environments:

1. Download models on a connected machine: ollama pull llama3.3:70b
2. Transfer model files to the air-gapped network (USB, sneakernet)
3. Start the private AI fleet — runs without internet as offline AI

The private AI router discovers nodes on the local network. No DNS, no external lookups, no cloud callbacks. True offline AI.

Private AI compliance#

Compliance NeedHow private AI helpsHIPAAPrivate AI processes PHI on your hardware onlyGDPRPrivate AI keeps all data in your jurisdictionData residencyPrivate AI — all processing on your hardwareNo subprocessorsPrivate AI uses no cloud APIs for inferenceAudit trailPrivate AI logs every request (model, node, latency)Data sovereigntyPrivate AI — your data, your machines, your controlData minimizationPrivate AI traces store routing metadata, never prompts

Monitor your private AI fleet#

bash

# Private AI fleet status
curl -s http://localhost:11435/fleet/status | python3 -m json.tool

# Private AI health checks
curl -s http://localhost:11435/dashboard/api/health | python3 -m json.tool

Private AI dashboard at http://localhost:11435/dashboard — accessible only on your local network.

Example private AI fleet response:

json

{
  "fleet": {"nodes_online": 3, "private_ai": true},
  "nodes": [
    {"node_id": "Private-AI-Server-1", "models_loaded": ["llama3.3:70b"]},
    {"node_id": "Private-AI-Server-2", "models_loaded": ["deepseek-r1:70b"]}
  ]
}

Full documentation#

• Agent Setup Guide
• Configuration Reference
• API Reference

Contribute#

Ollama Herd is open source (MIT). Private AI for everyone:

• Star on GitHub — help others discover private AI
• Open an issue — share your private AI setup
• PRs welcome — CLAUDE.md gives AI agents full context. 444 tests.

Guardrails#

• No automatic downloads — all private AI model pulls require explicit user confirmation.
• Private AI model deletion requires explicit user confirmation.
• No external network access — private AI router and nodes communicate only locally.
• Private AI traces never store prompt content — only routing metadata.
• Never delete or modify files in ~/.fleet-manager/.