Inference API (OpenAI-compatible)
Green Compute serves open models behind an OpenAI-compatible API. Any OpenAI SDK or tool works once you change the base URL and key.
- Base URL:
https://api.green-compute.com/v1 - Auth:
Authorization: Bearer <your API key>
List models
curl -s https://api.green-compute.com/v1/models -H "Authorization: Bearer $GC_KEY"
Use an id from this list verbatim as model. It lists the models your key can call right now: the shared models, and your own private models (and any shared with you), each only while a replica is ready to answer. A model that is starting, paused or down isn't listed; calling it anyway tells you which (see Errors). Per-token prices for the shared models are in GET https://api.green-compute.com/platform/pricing (public).
Chat completions
curl -s https://api.green-compute.com/v1/chat/completions \
-H "Authorization: Bearer $GC_KEY" -H "Content-Type: application/json" \
-d '{
"model": "<id from /v1/models>",
"messages": [{"role": "user", "content": "Say hello in five words."}],
"max_tokens": 200
}'
Python
from openai import OpenAI
client = OpenAI(base_url="https://api.green-compute.com/v1", api_key="YOUR_KEY")
resp = client.chat.completions.create(
model="<id from /v1/models>",
messages=[{"role": "user", "content": "Say hello in five words."}],
max_tokens=200,
)
print(resp.choices[0].message.content)
Streaming
Set "stream": true. Responses are standard server-sent events, ending with data: [DONE]. The final chunk carries usage. If you disconnect mid-stream, you are billed only for the tokens actually generated, and the model stops generating. A request that fails before the first token (refused, or the model is down) gets its error status and JSON body, never a 200 stream that breaks off.
A long answer is best streamed: a request that isn't streamed must finish within 120 seconds (longer for the largest reasoning models), or it ends with 504 upstream_timeout.
An account may have up to 8 requests in progress at once to the shared models (streams included, until they end); one more is answered at once with 429 rate_limited and Retry-After: 2. Send more as earlier ones finish. Requests to your own private models aren't limited this way: it's your GPU.
What a request may ask for
These are checked before the request reaches a model. A request outside them is answered at once with 422 validation_error, detail naming the field, and nothing is billed:
| Field | Allowed |
|---|---|
messages | at least one message, each with a role |
max_tokens, max_completion_tokens | 1 up to the model's context length when it is known, else 131072 |
n | 1 to 8 (each completion is billed) |
temperature | 0 to 2 |
top_p | above 0, at most 1 |
presence_penalty, frequency_penalty | -2 to 2 |
The model checks the rest itself, for example that the prompt plus max_tokens fits in its context. When it refuses a request, you get 400 invalid_request (or 413/422 when that is the model's status) with the model's own reason in error.message and its status in detail.upstream_status. That's a problem with the request: change it rather than retrying. It is not billed, and it doesn't count against the model for anyone else.
Tool calling
Standard OpenAI tools / tool_choice are passed through. tool_calls come back in the usual place on the message.
Letting the model decide when to call a tool (tool_choice: "auto", which is the default when you send tools, as is tool_choice: null) needs a model started with automatic tool choice. For one that wasn't, the API answers 400 invalid_request up front saying so: name the function in tool_choice ({"type": "function", "function": {"name": "..."}}), send tool_choice: "required", or leave tools out.
Reasoning models
Some models think before answering. For those:
- the thinking arrives in
message.reasoning_content(notreasoning) and the answer inmessage.content; - thinking counts against
max_tokens, so budget generously (2000+). Ifcontentcomes back empty, the model ran out of budget while thinking; raisemax_tokensand retry.
Legacy completions
POST /v1/completions takes a prompt (one string) instead of messages and answers in OpenAI's text_completion shape: object: "text_completion" and the text in choices[].text, streamed chunks likewise. The prompt reaches the model as a single user message, so the model's chat template applies: an instruction gets an answer rather than a raw continuation. suffix, echo, logprobs and best_of above 1 aren't supported (422); for log probabilities use chat completions. The same limits as above apply.
curl -s https://api.green-compute.com/v1/completions \
-H "Authorization: Bearer $GC_KEY" -H "Content-Type: application/json" \
-d '{"model": "<id from /v1/models>", "prompt": "Name three primes.", "max_tokens": 50}'
Embeddings
No embedding model is served yet. POST /v1/embeddings answers 400 invalid_request ("no embedding model is available"), or 404 model_not_found for a model name it doesn't know. It never returns vectors it made up.
Using it from coding tools
Any tool that accepts a custom OpenAI base URL works:
| Setting | Value |
|---|---|
| Base URL | https://api.green-compute.com/v1 |
| API key | your Green Compute key |
| Model | an id from /v1/models |
Your own models
A model you deploy yourself (Models → Deploy in the dashboard) also gets its own address, shown on its page:
https://<name>.inference.green-compute.com/v1
Use it as the base URL in any OpenAI client, with your API key as usual. Requests sent there always go to that model, so the model field doesn't decide where they're routed (send the model's name anyway). The same model also answers at https://api.green-compute.com/v1 with model set to its name.
The address is assigned when you create the model and stays the same across redeploys. It answers once a deployment is ready. A private model only accepts keys from your account and any account you've shared it with; the address doesn't make it public.
Your own model is charged by the hour while it runs, like a GPU rental ($0.40 an hour on one RTX 4090), and requests to it aren't charged per token. It pauses after 30 minutes with no requests; a paused model costs nothing. See Pricing. A request to a paused model answers 409 model_not_running, with the deployment to resume in detail.deployment_id (POST /platform/deployments/{id}/resume, or Resume on its page).
If your model stops answering its health checks while it runs (its model server crashed, or its machine went away), it is redeployed automatically, as a new deployment of the same model: the model keeps its address, name and id, and only the deployment is new. The failed deployment's last_error names the new one. That happens at most 3 times an hour, only for a model that had been serving, and only when a deploy by you would be allowed then: an hour of it in credit, under your account's limits (rentals at once, spend per 24 hours), and a GPU it fits on free. Otherwise last_error says why it wasn't restarted. While the new one starts, requests answer 409 model_not_ready; it isn't billed until it is ready.
curl https://<name>.inference.green-compute.com/v1/chat/completions \
-H "Authorization: Bearer $GC_KEY" -H "Content-Type: application/json" \
-d '{"model": "<model name>", "messages": [{"role": "user", "content": "Hello"}]}'
Over the API, the address is the workload's ingress_host, and a ready deployment's endpoint is its base URL.
A gated or private Hugging Face model needs a token. Rather than sending it as hf_token when you create the model, save it as a secret and pass its name: {"template": "vllm", "model": "meta-llama/Llama-3.1-8B-Instruct", "hf_token_secret": "hf-read"} on POST /platform/workloads. Only the name is stored with the model; the token goes to the machine that downloads the weights, and no response contains it.
POST /platform/workloads/{id}/warmup warms a model up: a short event stream, warmup_started then warmup_complete (it needs the rentals scope). The GET form of the same URL still answers, but is deprecated (it answers with a Deprecation: true header and a Link to the POST) and isn't in the OpenAPI spec.
Errors
Every error has the usual error object (code, message, retryable, retry_after_seconds; see Errors). For inference:
| Status | error.code | What happened | What to do |
|---|---|---|---|
400 | invalid_request | The model refused the request (its reason is in message), or it asks for something this model doesn't support, such as automatic tool choice. | Change the request. Not retryable. |
404 | model_not_found | No model by that name, id or address that your key may use. | Pick an id from GET /v1/models. |
409 | model_not_ready | The model has a replica starting and none ready yet. | Retry after Retry-After (30 s). |
409 | model_not_running | Your own model isn't running: paused (resume it) or its last deployment ended (deploy it again). | detail.deployment_id names it. Not retryable as is. |
422 | validation_error | A field is outside what a request may ask for. | detail names the field. |
502 | upstream_error | The model is up, but failed on this request (another replica is tried first, when there is one). | A shorter request or fewer tokens may work. Not retryable unchanged (x-should-retry: false). |
429 | rate_limited | Your account already has 8 requests in progress to shared models. | Retry after Retry-After (2 s), or as one finishes. |
503 | model_unavailable | No replica of the model is answering right now (its health check fails, its machine isn't reporting, or a new replica is being started), or every replica is at capacity. It comes back by itself. | Retry after Retry-After (30 s; 5 s when only the connection dropped; 2 s at capacity). |
504 | upstream_timeout | The model is up, but didn't finish this request in time. | Stream it, or ask for fewer tokens. Not retryable unchanged (x-should-retry: false). |
A request that fails with any of these isn't billed. A failure counts against the model only when its health check fails too, or, after it fails several requests in a row while that check passes, when the platform's own 1-token test request to it fails as well (that test is never billed); and a model is taken down only after that has gone on for 30 seconds. One bad or slow request, a moment's stall, or a short restart of its machine's agent never takes it down for anyone else.
Billing
Shared models: per token, input and output priced separately, with a small minimum charge per request. See GET /platform/pricing for the live figures.
Your own (private) models: by the hour while they run, billed per started minute, with no per-token charge. See Pricing.
Green Compute