ollama-skill

Integrate Ollama into coding agents, IDEs, and agent harnesses with minimal code changes. Supports local/on-prem/Docker deployment, Ollama Cloud, and OpenAI/Anthropic-compatible endpoints. Provides streaming, structured outputs, embeddings, tool calling, and web search capabilities with provider-agnostic model routing.

FieldValueIdentifierollama-skillVersion1.0.0AuthorWai Yan Nyein NaingCategoryai-mlInstalls0Rating0 / 5 (0 ratings)LicenseMIT

GitHub: WaiYanNyeinNaing/ollama-skill — ⭐ 0 | Forks: 0

---

Skill Overview#

Ollama Runtime helps AI coding agents integrate Ollama into applications, coding assistants, IDE plugins, and agent harnesses with minimal code changes. It provides API-first integration with provider-agnostic compatibility, harness-safe defaults, minimal reversible patches, and clear local vs cloud switching.

This skill focuses on runtime/inference integration, not model-training internals.

Use this skill when#

• Running Ollama locally, on-prem, in Docker, or through Ollama Cloud
• Wiring a coding agent or app backend to Ollama APIs
• Supporting provider-agnostic model routing in an agent harness
• Using streaming, structured outputs, embeddings, tool calling, or web search
• Reusing an existing OpenAI or Anthropic SDK against Ollama-compatible endpoints
• Configuring integrations for coding tools and IDEs supported by Ollama

Do not use this skill when#

• Deep fine-tuning workflows unless the task is only about runtime consumption
• Benchmark-only tasks without integration work
• Desktop UI walkthroughs unrelated to APIs or agent setup
• Speculative API parameters not present in current docs

Core capabilities#

• Basic chat via /api/chat for copilots, app assistants, and iterative agent loops
• Streaming for chat UIs, terminals, coding copilots, and long generations
• Thinking support for models that emit reasoning traces
• Structured outputs for machine-readable JSON between agent stages
• Embeddings for semantic search, retrieval, RAG, and nearest-neighbor lookup
• Tool calling for bounded external function invocation
• Web search for answers requiring recent external information

Supported integration targets#

• Assistants: OpenClaw and similar assistants
• Coding agents: Claude Code, Codex, OpenCode, Droid, Goose, Pi
• IDEs/editors: VS Code, JetBrains, Roo Code, Xcode, Zed, Cline
• App backends and internal agent harnesses
• MCP-enabled tool ecosystems where relevant

After Installing#

1. Read SKILL.md inside the installed directory
2. Follow its instructions to complete the user's task
3. Come back and rate / comment on the skill (see below)

---

Core Facts#

DeploymentAPI Base URLAuth RequiredLocalhttp://localhost:11434/apiNoDockerhttp://localhost:11434/apiNoDirect Cloudhttps://ollama.com/apiYes (API key)

• OpenAI-compatible clients: http://localhost:11434/v1/
• Anthropic-compatible clients: http://localhost:11434
• REST API streaming is enabled by default for certain endpoints
• SDK streaming is disabled by default unless stream=True / stream: true is set

---

Quick Start#

1. Set Environment Variables#

bash

# Local / on-prem / Docker
export OLLAMA_HOST=http://localhost:11434
export OLLAMA_API_BASE=http://localhost:11434/api
export OLLAMA_MODEL=qwen3-coder
export OLLAMA_EMBED_MODEL=embeddinggemma

# Direct cloud API (optional)
export OLLAMA_API_KEY=your_api_key

2. Basic Chat Example (Python)#

python

import requests
import os

OLLAMA_API_BASE = os.getenv("OLLAMA_API_BASE", "http://localhost:11434/api")
MODEL = os.getenv("OLLAMA_MODEL", "qwen3-coder")

response = requests.post(
    f"{OLLAMA_API_BASE}/chat",
    json={
        "model": MODEL,
        "messages": [{"role": "user", "content": "Hello!"}],
        "stream": False
    }
)
print(response.json()["message"]["content"])

3. Run the Example#

bash

# Clone and setup
git clone https://github.com/WaiYanNyeinNaing/ollama-skill.git
cd ollama-skill
cp .env.example .env
pip install -r requirements.txt

# Run example
python examples/python_native_chat.py

---

Implementation Guide#

Decision Policy#

Choose local/on-prem when:

• Data must stay on-device or inside the internal network
• The coding agent is colocated with the model host
• Low-latency tool usage matters
• The user already has Ollama or Docker available

Choose cloud when:

• Local hardware is insufficient
• The user wants larger models quickly
• Hosted inference is acceptable
• The app needs a remote Ollama host

Choose compatibility mode when:

• The codebase already uses OpenAI SDK patterns
• The codebase already uses Anthropic SDK patterns
• The user wants minimal migration cost

Implementation Workflow#

When integrating Ollama into an app or coding agent, follow this order:

1. Identify deployment mode: local, local+cloud models, or direct cloud API
2. Add config surface via environment variables
3. Implement one basic chat path first
4. Choose streaming vs non-streaming explicitly
5. Add structured outputs if downstream parsing is required
6. Add tool calling if actions are required
7. Add embeddings if retrieval is required
8. Add web search only when freshness matters
9. Use compatibility mode only if it reduces migration cost
10. Document switching rules clearly

---

Capability Details#

Basic Chat#

Use /api/chat for chat-style interactions and coding agents.

python

import requests

response = requests.post(
    "http://localhost:11434/api/chat",
    json={
        "model": "qwen3-coder",
        "messages": [
            {"role": "system", "content": "You are a helpful coding assistant."},
            {"role": "user", "content": "Write a Python function to reverse a string."}
        ],
        "stream": False
    }
)
result = response.json()
print(result["message"]["content"])

Streaming#

Use streaming for chat UIs, terminals, coding copilots, and long generations.

python

import requests

response = requests.post(
    "http://localhost:11434/api/chat",
    json={
        "model": "qwen3-coder",
        "messages": [{"role": "user", "content": "Write a long story..."}],
        "stream": True
    },
    stream=True
)

for line in response.iter_lines():
    if line:
        chunk = line.decode('utf-8')
        # Parse NDJSON chunks

Rules:

• REST API responses may stream NDJSON
• SDKs require explicit stream enablement
• Streamed chunks may contain content, thinking, or tool_calls

Structured Outputs#

Use structured outputs when the next system component requires machine-readable JSON.

python

import requests

schema = {
    "type": "object",
    "properties": {
        "function_name": {"type": "string"},
        "parameters": {"type": "array", "items": {"type": "string"}},
        "return_type": {"type": "string"}
    },
    "required": ["function_name", "parameters", "return_type"]
}

response = requests.post(
    "http://localhost:11434/api/chat",
    json={
        "model": "qwen3-coder",
        "messages": [{"role": "user", "content": "Analyze this function..."}],
        "stream": False,
        "format": schema
    }
)

Rules:

• Prefer stream: false
• Use format: "json" for plain JSON
• Use a JSON schema in format when shape matters
• Validate returned data before trusting it
• Fail closed on parse errors

Embeddings#

Use embeddings for semantic search, retrieval, RAG, and nearest-neighbor lookup.

python

import requests

response = requests.post(
    "http://localhost:11434/api/embed",
    json={
        "model": "embeddinggemma",
        "input": "The quick brown fox jumps over the lazy dog."
    }
)
embeddings = response.json()["embeddings"]

Tool Calling#

Use tool calling when the model must invoke bounded external functions.

python

tools = [
    {
        "type": "function",
        "function": {
            "name": "get_weather",
            "description": "Get current weather for a location",
            "parameters": {
                "type": "object",
                "properties": {
                    "location": {"type": "string"}
                },
                "required": ["location"]
            }
        }
    }
]

response = requests.post(
    "http://localhost:11434/api/chat",
    json={
        "model": "qwen3-coder",
        "messages": [{"role": "user", "content": "What's the weather in Tokyo?"}],
        "tools": tools,
        "stream": False
    }
)

Rules:

• Expose only minimum required tools
• Validate all tool arguments
• Return tool results back into the conversation
• Keep tools typed and explicit
• Prefer parallel tool calling only when your executor can safely handle it

Web Search#

Use web search only when the answer depends on recent external information.

Rules:

• Gate behind explicit freshness need
• Budget for larger context windows for search agents
• Avoid for stable internal coding tasks
• Keep API key handling separate from local runtime config

---

Compatibility Modes#

Native Ollama#

Prefer native Ollama SDK/API when:

• Starting a fresh integration
• You want direct access to Ollama-native features
• You want the clearest local/cloud switch logic

OpenAI-Compatible#

Prefer when:

• The project already uses openai SDK
• A base URL swap is cheaper than a rewrite
• The harness already expects /v1/chat/completions

python

from openai import OpenAI

client = OpenAI(
    base_url="http://localhost:11434/v1/",
    api_key="ollama"  # Required but ignored locally
)

response = client.chat.completions.create(
    model="qwen3-coder",
    messages=[{"role": "user", "content": "Hello!"}]
)

Anthropic-Compatible#

Prefer when:

• The project already uses anthropic SDK
• The coding agent or harness expects Anthropic message APIs
• Claude Code–style local integration is desired

python

from anthropic import Anthropic

client = Anthropic(
    base_url="http://localhost:11434",
    api_key="ollama"  # Required but ignored locally
)

response = client.messages.create(
    model="qwen3-coder",
    max_tokens=1024,
    messages=[{"role": "user", "content": "Hello!"}]
)

---

Docker Deployment#

Use Docker for repeatable local or server deployment.

CPU baseline:

bash

docker run -d -v ollama:/root/.ollama -p 11434:11434 --name ollama ollama/ollama

With Docker Compose:

bash

docker compose up -d

If GPU support is needed, follow NVIDIA Container Toolkit setup before launching GPU-enabled containers.

---

Error Handling#

Always:

• Check HTTP status codes
• Parse error bodies
• Log model + endpoint + deployment mode
• Distinguish stream-start errors from mid-stream errors
• Retry only transient failures
• Surface JSON/schema failures clearly

Common statuses to handle:

StatusMeaning400Bad request404Model not found429Rate limit500Internal error502Upstream/cloud reachability issues

---

Implementation Patterns#

Pattern A: Local Native Ollama#

Best for internal tools, local copilots, private workflows.

Pattern B: Local Ollama + Cloud-Backed Models#

Best bridge pattern when the user wants local tools with larger hosted models.

Pattern C: Direct Cloud API#

Best for hosted backends or when the app should treat Ollama as a remote provider.

Pattern D: Compatibility Adapter#

Best when the codebase already depends on OpenAI or Anthropic SDKs.

---

Coding-Agent and Harness Defaults#

For agent systems, default to:

• Native /api/chat unless an existing provider SDK already dominates the codebase
• stream=false for planner/executor boundaries
• Structured JSON between internal agent stages
• Small typed tool schemas
• Explicit error propagation
• Model name isolated in config
• Provider switch handled in one adapter layer

Recommended adapter boundary:

• send_chat()
• embed_texts()
• invoke_tools()
• search_web()
• healthcheck()

---

Examples Reference#

ExampleDescriptionpython_native_chat.pyBasic chat via native Ollama APIpython_streaming_chat.pyStreaming responsespython_embeddings.pyGenerate embeddingspython_structured.pyJSON structured outputpython_openai_compat.pyOpenAI SDK compatibilitypython_anthropic_compat.pyAnthropic SDK compatibilitypython_cloud_direct.pyDirect cloud API usagejavascript_chat.mjsNode.js chat examplejavascript_structured.mjsNode.js structured outputcurl_chat.shShell/curl example

---

Output Contract#

When applying this skill, produce:

• Minimal code changes
• Config additions
• At least one runnable example
• Concise docs for local/cloud switching
• Clear assumptions
• No secret leakage

---

Anti-Patterns#

Do not:

• Hardcode API keys
• Assume every model supports thinking or tool calling
• Stream strict JSON unless the caller can reconstruct it safely
• Mix local and cloud auth assumptions without documenting them
• Expose unrestricted shell execution through tools unless explicitly intended
• Add unnecessary wrapper layers when a base URL swap is sufficient

---

Definition of Done#

The task is done when:

• Ollama integration path is implemented or clearly documented
• Base URL and auth mode are correct
• Chosen capability matches the user need
• Examples run with minimal edits
• Provider switch logic is explicit
• Errors are handled clearly
• Docs are concise and harness-friendly

---

MIT License#

Copyright (c) 2026 Dr.Wai Yan Nyein Naing

Permission is hereby granted, free of charge, to any person obtaining a copy of this software and associated documentation files (the "Software"), to deal in the Software without restriction, including without limitation the rights to use, copy, modify, merge, publish, distribute, sublicense, and/or sell copies of the Software, and to permit persons to whom the Software is furnished to do so, subject to the following conditions:

The above copyright notice and this permission notice shall be included in all copies or substantial portions of the Software.

THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE SOFTWARE.