Guides

Python guide

Make an authenticated, server-side request to the LLMFLUX gateway.

Set up

Create an API key in the dashboard, complete any required Telegram verification, and ensure the account has credit. Keep the key on your server.

python -m pip install openai
# For Messages: python -m pip install anthropic

export LLMFLUX_API_KEY="YOUR_GATEWAY_KEY"
export LLMFLUX_MODEL="YOUR_MODEL_ID"

Discover models

Choose a compatible enabled ID from the authenticated model list, then replace YOUR_MODEL_ID or set LLMFLUX_MODEL. Do not assume a model name from another provider is available unchanged.

curl https://api.llmflux.dev/v1/models \
  -H "Authorization: Bearer $LLMFLUX_API_KEY"

Send a request

import os
from openai import OpenAI

client = OpenAI(api_key=os.environ["LLMFLUX_API_KEY"], base_url="https://api.llmflux.dev/v1")
response = client.chat.completions.create(model=os.environ["LLMFLUX_MODEL"], messages=[{"role": "user", "content": "Hello from LLMFLUX"}], stream=False)
print(response.choices[0].message.content)

A successful non-streaming chat request returns choices[].message and provider usage. See Chat completions for the full compatibility boundary.

Use Responses or Messages

The OpenAI SDK also exposes Responses. For Anthropic-format Messages use the Anthropic SDK and its root base URL.

import os
from openai import OpenAI

client = OpenAI(api_key=os.environ["LLMFLUX_API_KEY"], base_url="https://api.llmflux.dev/v1")
response = client.responses.create(model=os.environ["LLMFLUX_MODEL"], input="Hello from LLMFLUX", stream=False)
print(response.output_text)

import os
from anthropic import Anthropic

client = Anthropic(api_key=os.environ["LLMFLUX_API_KEY"], base_url="https://api.llmflux.dev")
message = client.messages.create(model=os.environ["LLMFLUX_MODEL"], max_tokens=1024, messages=[{"role": "user", "content": "Hello from LLMFLUX"}])
print(message.content)

Stream output

import os
from openai import OpenAI

client = OpenAI(api_key=os.environ["LLMFLUX_API_KEY"], base_url="https://api.llmflux.dev/v1")
response = client.chat.completions.create(model=os.environ["LLMFLUX_MODEL"], messages=[{"role": "user", "content": "Hello from LLMFLUX"}], stream=True)
for event in response:
    print(event)

Handle errors during iteration as well as when starting the request. See Streaming for event schemas and usage limitations.

Inspect errors and usage

SDK errors expose HTTP status and the response body. Use gateway error codes to distinguish invalid keys, exhausted credit and retryable failures. SDKs may retry automatically: configure bounded retries and remember inference has no gateway idempotency guarantee.

curl https://api.llmflux.dev/v1/usage \
  -H "Authorization: Bearer $LLMFLUX_API_KEY"

Usage and remaining credits are account-wide, even when queried with one key.