Skip to content

For the complete documentation index, see llms.txt.

Add your own model provider keys

Docs evals use the inference providers your organization sets up on the Models settings page. Use the AI your company already has: add a provider API key or your own endpoint. If you self-host Promptless, set your internal inference here too.

Inference options vary by plan. To talk through the setup that fits your organization, contact the Promptless team.

Promptless works with any endpoint that serves an OpenAI-compatible or Anthropic-compatible API. Anthropic, OpenAI, and OpenRouter are built in, so each needs only an API key. Your organization holds one key for each built-in provider and any number of custom endpoints.

  • An organization admin role. Other members see the page read-only.
  • An API key for each provider.
  1. Open Settings, then Models.

  2. On the provider’s card, select Add OpenRouter key, Add Anthropic key, or Add OpenAI key.

  3. Enter the API key, then select Add key.

  4. Select Run test call. The card shows Verified, or Test call failed with the provider’s message.

The test call sends one short prompt to the lowest-priced model the key reaches. Promptless encrypts the key on save. After that, the dashboard shows only a short fingerprint.

The endpoint must use https and resolve only to public IP addresses. Promptless doesn’t follow redirects.

  1. On the Models settings page, select Add custom endpoint.

  2. Enter a Provider ID of lowercase letters, digits, and hyphens, such as dashscope. It can’t match a built-in provider’s ID.

  3. Enter a Display name, the Base URL, and the API key.

  4. Under API, select OpenAI chat completions or Anthropic messages.

  5. Add a row for each model with its Model ID, context window, and maximum output tokens. Promptless enforces a spending limit on each run, so you must also enter what the endpoint charges. Enter its input and output prices in USD per million tokens. Use 0 for tokens the endpoint doesn’t charge for.

  6. Select Add endpoint, then Run test call on the endpoint’s card.

To change the model list later, select Edit models on the endpoint’s card.

The Models you can pick table lists each model your enabled keys reach, with its price per million tokens and its model ref. A ref has the form <provider-id>/<model-id>, such as openrouter/qwen/qwen3-coder or anthropic/claude-haiku-4-5.

A model shows Ready once its provider passes a test call. Until then it shows Verify provider first, and runs can’t use it.

ActionControl on the provider’s cardEffect
ReplaceReplace keyThe provider is unverified until you run the test call again.
DisableDisable, then EnableRuns can’t use the provider’s models while it’s disabled.
RemoveThe delete icon, then confirmRuns can’t use the provider’s models.

Editing a custom endpoint’s model list also resets it to unverified.