vultr-inference#

Generate images and text using Vultr's Inference API.

Setup#

Uses the same API key as Vultr Cloud API. Store it at:

text

~/.config/vultr/api_key

Image Generation#

Available Models#

ModelDescriptionflux.1-devFLUX.1-dev - High qualityflux.1-schnellFLUX.1-schnell - Fast generationstable-diffusion-3.5-mediumSD 3.5 Medium - Balanced

Generate Image#

bash

curl -X POST "https://api.vultrinference.com/v1/images/generations" \
  -H "Authorization: Bearer $VULTR_API_KEY" \
  -H "Content-Type: application/json" \
  -d '{
    "model": "flux.1-schnell",
    "prompt": "a hedgehog eating a burger in Amsterdam",
    "n": 1,
    "size": "1024x1024"
  }'

Parameters#

ParameterTypeDescriptionmodelstringflux.1-dev, flux.1-schnell, stable-diffusion-3.5-medium``promptstringText description of imagenintNumber of images (1-4)sizestring256x256, 512x512, 1024x1024``response_formatstringurl (default) or b64_json

Response#

json

{
  "created": 1734567890,
  "data": [
    {
      "url": "https://ewr.vultrobjects.com/vultrinference-images/tmp_xxx.png"
    }
  ]
}

Text Generation (Chat Completions)#

Available Models#

• llama-3.1-405b-instruct - Meta Llama 3.1 405B
• llama-3.1-70b-instruct - Meta Llama 3.1 70B
• llama-3.1-8b-instruct - Meta Llama 3.1 8B
• mixtral-8x7b-32768 - Mixtral 8x7B
• qwen-2-72b-instruct - Qwen 2 72B

Chat Completion#

bash

curl -X POST "https://api.vultrinference.com/v1/chat/completions" \
  -H "Authorization: Bearer $VULTR_API_KEY" \
  -H "Content-Type: application/json" \
  -d '{
    "model": "llama-3.1-70b-instruct",
    "messages": [
      {"role": "user", "content": "Hello, how are you?"}
    ],
    "max_tokens": 100
  }'

Parameters#

ParameterTypeDescriptionmodelstringModel ID from list abovemessagesarrayChat messages with role and contentmax_tokensintMaximum tokens to generatetemperaturefloatRandomness (0-2, default 1)streamboolStream response (default false)

Python Example#

python

import os
import requests

API_KEY = open(os.path.expanduser("~/.config/vultr/api_key")).read().strip()

# Generate image
response = requests.post(
    "https://api.vultrinference.com/v1/images/generations",
    headers={"Authorization": f"Bearer {API_KEY}"},
    json={
        "model": "flux.1-schnell",
        "prompt": "a hedgehog eating a burger",
        "size": "512x512",
        "n": 1
    }
)

result = response.json()
image_url = result["data"][0]["url"]
print(f"Image URL: {image_url}")

# Download image
img_response = requests.get(image_url)
with open("generated_image.png", "wb") as f:
    f.write(img_response.content)

List Available Models#

bash

curl -s "https://api.vultrinference.com/v1/models" \
  -H "Authorization: Bearer $VULTR_API_KEY" | jq

Troubleshooting#

401 Unauthorized

• Check API key is valid
• Ensure key has inference permissions

400 Bad Request

• Check model name is correct
• Check size is valid (256x256, 512x512, 1024x1024)
• Check prompt is not empty

Rate Limits

• Default: 60 requests per minute
• Contact support for higher limits