Best practices

This page collects what to set up before you run Glexir in production: keys, limits, safe retries, streaming, cost control and monitoring. Each section links to the reference page with the full detail.

A production-ready client looks like this:

import Glexir from "@glexir/sdk";

export const glexir = new Glexir({
  apiKey: process.env.GLEXIR_API_KEY, // one key per environment
  maxRetries: 3,
  timeout: 120_000, // per attempt, in milliseconds
});

Use one key per environment

Create a separate API key for each environment and service, for example web-production, worker-production and staging. Then you can:

• revoke one key without taking down the others,
• see usage per key on the dashboard's Usage page,
• give each key its own monthly spend limit.

Keep keys on the server. Load them from GLEXIR_API_KEY or your secret manager, never from client-side code or a repository. A revoked key can keep working for up to 30 seconds.

@glexir/sdk reads GLEXIR_API_KEY when you don't pass apiKey. It calls https://api.glexir.com/v1 by default, so you only set GLEXIR_BASE_URL to reach a local gateway or another deployment.

GLEXIR_API_KEY=glx_live_...

Set spend limits

Glexir has two monthly spend limits. Both reset at the start of each calendar month.

A limit is checked before the request runs, against what you have spent this month plus what in-flight requests have reserved. Give experimental and batch keys a low limit so a bug can't drain your balance.

Requests per minute and concurrent requests are limited per organization, not per key, by your account tier. See Rate limits & spend limits.

Make retries safe with idempotency keys

A retry after a dropped connection can run a request twice. Glexir prevents this with the Idempotency-Key header: a POST with a key it has already seen gets the stored response instead of running, and being billed, again.

@glexir/sdk generates a key for every non-GET call and reuses it across that call's retries. You don't need to do anything for SDK retries.

Pass your own idempotencyKey when the same logical request can be sent by separate calls, for example a queue job that may be delivered twice or a worker that may restart:

import Glexir from "@glexir/sdk";

const glexir = new Glexir({ apiKey: process.env.GLEXIR_API_KEY });

async function summarize(jobId: string, text: string): Promise<string> {
  const { data, response } = await glexir.responses
    .create(
      { model: "gpt-5.4-mini", instructions: "Summarize in two sentences.", input: text },
      { idempotencyKey: `summarize-${jobId}` },
    )
    .withRawResponse();

  if (response.headers.get("idempotent-replayed") === "true") {
    console.log(`job ${jobId}: served from the idempotency store`);
  }
  return data.output_text;
}

How keys behave:

• Scope and lifetime. Keys are scoped to your organization and remembered for 24 hours. At most 255 characters.
• Only POST requests use the key. GET and DELETE ignore it.
• Same request only. Reusing a key with a different method, path or body returns 422 idempotency_key_reused.
• In progress. If the first request is still running, a retry gets 409 idempotency_request_in_progress. Retry later with the same key.
• Only successes are stored. If the first attempt failed with any error, including 429 and 5xx, the key is released and the next attempt runs normally. This means you can retry a 402 with the same key after adding credit.
• Streams are not stored. A key used for a streaming request can't be replayed; a retry gets 409 idempotency_stream_replay.
• Uploads. For multipart requests (file uploads and transcriptions) the key is matched on the body size, not its content. Use a new key for every file.

With the openai packages, send the header yourself. See Use any OpenAI SDK.

Retry the right errors

@glexir/sdk retries automatically:

• What: connection errors, timeouts, and 408, 409, 429 and 5xx responses. A response header x-should-retry: true or false overrides this.
• How often: maxRetries times, 2 by default.
• How long it waits: 0.5 s, then 1 s, 2 s and so on up to 8 s, minus up to 25% jitter. When the response has a retry-after-ms or retry-after header of 60 seconds or less, it waits that long instead.

Override the client defaults per request with RequestOptions, the last argument of every method:

import Glexir from "@glexir/sdk";

const glexir = new Glexir({ apiKey: process.env.GLEXIR_API_KEY });

const embedding = await glexir.embeddings.create(
  { model: "text-embedding-3-small", input: "fast path" },
  {
    maxRetries: 0, // fail fast on this call
    timeout: 10_000, // per attempt, in milliseconds
    signal: AbortSignal.timeout(15_000), // cancels the call and any retries
    headers: { "x-trace-id": "abc123" }, // extra header; null removes a default one
  },
);
console.log(embedding.data.length);

Handle these in your own code instead of retrying:

The SDK also retries every 409, which only helps for idempotency_request_in_progress. See HTTP error codes.

Stream long outputs

Stream when a person is waiting for the output or when outputs are long. The helpers accumulate the result for you:

import Glexir from "@glexir/sdk";

const glexir = new Glexir({ apiKey: process.env.GLEXIR_API_KEY });

const stream = glexir.responses.stream({
  model: "gpt-5.4-mini",
  input: "Explain idempotency in one paragraph.",
  max_output_tokens: 800,
});

for await (const event of stream) {
  if (event.type === "response.output_text.delta") process.stdout.write(event.delta);
}

const final = await stream.finalResponse();
console.log("\n", final.usage?.output_tokens, "output tokens");

• glexir.responses.stream() returns a ResponseStream with finalResponse(), finalText(), outputText, currentResponse() and abort().
• glexir.chat.completions.stream() returns a ChatCompletionStream with finalChatCompletion(), finalContent(), snapshot() and abort().
• create({ ..., stream: true }) returns the raw Stream of events. It has tee() to split it, toReadableStream() to relay it as newline-delimited JSON, and controller to abort it.

Things to know:

• Errors mid-stream arrive as an APIError with status undefined. Check err.code (stream_timeout, upstream_stream_error, or an upstream code).
• No automatic retry once a stream has started, and a stream can't be replayed from the idempotency store. Decide in your code whether to send a new request.
• Billing continues for tokens already generated if you stop reading early. Usage is settled from what the upstream reports, or from an input estimate if the stream ended without a report.
• Usage on Chat Completions streams is only included in the stream when you set stream_options: { include_usage: true }.
• An upstream stream that sends nothing for the gateway's idle timeout (10 minutes by default) is closed with stream_timeout.

See Streaming.

Control cost

Cap output tokens

Set max_output_tokens (Responses) or max_completion_tokens (Chat Completions). It limits what you spend, and it shrinks the credit reserved while the request runs. Without a cap, the gateway reserves for 32,000 output tokens per request, which can cause 402 credit_balance_exhausted on a small balance or many parallel requests.

Count tokens before you send

glexir.responses.inputTokens.count() returns the input token count for a request without running it, so you can trim long inputs first. It takes the same input fields as responses.create and is not billed.

import Glexir from "@glexir/sdk";

const glexir = new Glexir({ apiKey: process.env.GLEXIR_API_KEY });

const { input_tokens } = await glexir.responses.inputTokens.count({
  model: "gpt-5.4-mini",
  input: "How many tokens is this?",
});
console.log(input_tokens);

Use batches for work that can wait

Batch requests are billed at half the standard rate and finish within 24 hours. Use them for evaluations, backfills, classification and embedding jobs. toBatchJSONL builds the input file:

import Glexir, { toBatchJSONL } from "@glexir/sdk";

const glexir = new Glexir({ apiKey: process.env.GLEXIR_API_KEY });

const file = await glexir.files.create({
  purpose: "batch",
  file: toBatchJSONL([
    { custom_id: "t1", url: "/v1/responses", body: { model: "gpt-5.4-mini", input: "Classify: great product" } },
    { custom_id: "t2", url: "/v1/responses", body: { model: "gpt-5.4-mini", input: "Classify: arrived broken" } },
  ]),
});

const batch = await glexir.batches.create({
  input_file_id: file.id,
  endpoint: "/v1/responses",
  completion_window: "24h",
});
console.log(batch.id, batch.status);

The whole batch's estimated cost is reserved when you create it, and you are charged for the lines that complete. See Batches.

Use the flex tier for slow, non-urgent calls

Set service_tier: "flex" on Responses or Chat Completions to run a text request at the flex rate, which is half the standard rate in the current price book. Flex requests can be slower. Flex is available on every account tier for text models, but not for embeddings.

import Glexir from "@glexir/sdk";

const glexir = new Glexir({ apiKey: process.env.GLEXIR_API_KEY });

const response = await glexir.responses.create({
  model: "gpt-5.4-mini",
  input: "Draft a changelog entry.",
  service_tier: "flex",
});
console.log(response.service_tier, response.output_text);

You are billed for the tier that actually served the request, which is reported in service_tier.

Get prompt cache hits

Cached input tokens are billed at the lower cached input rate shown on Pricing. To get more hits:

• Put the parts that don't change (instructions, tool definitions, examples) at the start of the input, and the variable part at the end.
• Send the same prompt_cache_key for requests that share a prefix. Glexir namespaces the key to your organization, so it never matches another customer's cache.
• Check usage.input_tokens_details.cached_tokens to see how many input tokens were cached.

On GPT-5.6 and later models, writing to the cache costs more than uncached input. Caching pays off when a prefix is reused.

Buy credit in larger top-ups

Usage is billed at OpenAI's list price. A larger single top-up adds bonus credit on top of what you pay:

The bonus part is non-refundable promo credit. It is spent before purchased credit and expires with the purchase. See Pricing and Credits & billing.

Monitor usage and errors

• Log request IDs. Every response has an x-request-id header. The SDK puts it on parsed objects as _request_id, on errors as err.requestID, and on .withRawResponse() results as requestId. Quote it when you contact support.
• Log error codes. Record err.status and err.code so you can alert on 402 and on rising 429 or 5xx rates.
• Watch the dashboard. The Usage page breaks down usage by API key, model and endpoint, and can export CSV.
• Avoid running dry. Turn on auto-recharge on the Billing page so a low balance tops up before requests start failing with 402.

import Glexir, { APIError } from "@glexir/sdk";

const glexir = new Glexir({ apiKey: process.env.GLEXIR_API_KEY });

try {
  const { data, requestId } = await glexir.responses
    .create({ model: "gpt-5.4-mini", input: "ping" })
    .withRawResponse();
  console.log(requestId, data.usage?.total_tokens);
} catch (err) {
  if (err instanceof APIError) {
    console.error(JSON.stringify({ status: err.status, code: err.code, requestId: err.requestID }));
  }
  throw err;
}