Manage LLM gateways

An LLM gateway is a language-model endpoint registered once for the whole platform. Platform services that use a language model — today Sentinel — ask the platform for a gateway every time they run, instead of carrying their own URL and API key.

Gateways are managed under Settings → Platform Settings, on the LLM gateways tab. Which gateway and model each Sentinel stage uses is set on the Sentinel settings tab. Both tabs are shown only to principals who can read gateways — see Permissions.

What can be a gateway

Any server with an OpenAI-compatible API: LiteLLM, vLLM, llama.cpp, Ollama, or a hosted provider that offers one. The platform checks a gateway with GET /v1/models, and Sentinel sends its requests to /v1/chat/completions, both under the gateway's base URL.

Create a gateway

1. Open the LLM gateways tab and choose Add gateway.

2. Fill in Add LLM gateway:

   Field	What to enter
   Name	A display name, for example LiteLLM (prod). At most 100 characters
   Slug	The name services ask for the gateway by, suggested from the name. 1–40 lowercase letters, digits and hyphens, starting with a letter or digit, and not used by another gateway. It cannot be changed later
   Base URL	The server root, for example https://litellm.example.com. A trailing /v1 or /v1/chat/completions is removed. See Base URL rules
   API key	Optional. Leave it empty for a gateway without authentication
   Set as default gateway	Services that do not name a gateway get the default one. The first gateway you add becomes the default whatever you choose here

3. Choose Test connection to try the URL and key before saving — see Test a connection.

4. Choose Add gateway.

The platform checks the gateway when you save and keeps it whatever the check finds: a gateway that is down right now is still registered, and the message after saving says what the check found.

Important

The API key is write-only. It is stored encrypted and sent to the gateway as a Bearer token, and neither the portal nor the admin API ever shows it again. The gateway's details and the edit sheet say only Key set or No key. The key is handed out in plain text over the platform's internal channel to any principal holding kernel/llmGateways/resolve at the root — Sentinel's service account, and also Platform Owners and Platform Contributors, whose */* includes it. Every hand-out is recorded, see What is recorded.

Base URL rules

The base URL must be an absolute http:// or https:// URL of at most 2048 characters, and must not contain:

• credentials, as in https://user:password@host — set the API key instead;
• a query string or a fragment;
• a link-local address such as 169.254.169.254;
• a loopback or unspecified address such as localhost, 127.0.0.1 or 0.0.0.0, which from a platform service is that service's own pod. A platform API running in the Development environment accepts these.

Private addresses are accepted, so a gateway running inside the cluster can be registered. A host name is not resolved when you save.

Test a connection

Test connection calls GET /v1/models under the base URL, with the key, and reports one of:

The gateway list

Each row shows the name, with a Default badge on the default gateway; the slug; the base URL; the health, with when it was last checked and the last error if the check failed; and how many of the gateway's models are enabled. Health is one of the results above, or Not checked for a gateway that has not been checked.

The platform checks every gateway again every 15 minutes by default, and updates its health and model list. The last error quotes the start of the server's answer, with the key removed. Health is for you to read: services are handed a gateway whatever its health, and their own request is the real test.

Choose which models services may use

Choose a gateway in the list to open its details. Models lists every model the gateway has; only the Enabled ones are offered to services.

• The first time a gateway lists its models, all of them are enabled. Models that appear in later checks arrive disabled, so you decide whether services may use them.
• A model the server stops listing is marked Missing. It is not removed and keeps its setting: an enabled model marked Missing is still offered to services until you disable it.
• Add model by id adds a model the server does not list. It is enabled at once and marked Manual. Manual models can be removed; a model the server has listed cannot, even once it is marked Missing, and a manual model the server starts listing stops being manual. A model id is at most 200 characters, without spaces.
• Each change saves the gateway's whole model list, and a saved list can hold at most 500 models. The checks add every model the server lists without that limit, so on a gateway that lists more than 500 models every change here is refused.

Every change is saved at once. A model a Sentinel stage is assigned to cannot be disabled or removed; the error names the stage's purpose, for example sentinel.brain. Assign that stage another model first — see Choose the gateway and model for each Sentinel stage.

Edit a gateway

In the gateway's details, choose Edit. You can change the Name, the Base URL, the API key and Set as default gateway; the slug is fixed. Changing the base URL or the key checks the gateway again when you save, and that check updates the model list as the periodic check does: models the server now lists arrive disabled, and models it no longer lists are marked Missing but stay enabled. After pointing a gateway at another server, review its Models.

The key field shows Key set or No key and keeps the stored key unless you choose:

• Replace key, or Add key when none is set, to enter a new key;
• Remove key, to remove the stored key when you save;
• Keep current key, to go back to keeping it.

Test connection in the edit sheet uses the stored key unless you replaced or removed it.

Move a gateway to another host

A stored key is only ever sent to the address it was saved for: the same scheme, host and port. When the new base URL points anywhere else, the edit sheet asks for the key again. Enter it for the new host, or choose Remove key if that host needs none; until you do, Test connection is disabled and the change cannot be saved. Changing only the path, for example adding /litellm, keeps the stored key.

Check a gateway again

Choose Re-verify in the gateway's details. The gateway is checked at once, and its health and model list are updated as by the periodic check.

Change the default gateway

Choose Set as default in the details of the gateway that should become the default, or tick Set as default gateway when you add or edit it. The previous default stops being one. The default gateway has no Set as default button and its checkbox cannot be cleared: to change the default, make another gateway the default.

Delete a gateway

In the gateway's details, choose Delete and confirm with Delete gateway. Services that ask for this gateway stop getting it from their next run. This cannot be undone.

The platform refuses the delete, and the dialog says why, when:

• the gateway is the default and other gateways exist — make another one the default first. The only gateway can be deleted even though it is the default;
• a Sentinel stage is assigned to it — the error names the stages. Assign them another gateway, or clear their assignment, first.

Choose the gateway and model for each Sentinel stage

Open the Sentinel settings tab. Model assignments has one row per Sentinel stage:

For each stage you want to set:

1. Choose a Gateway. Not assigned (use default gateway) clears the assignment.
2. Choose a Model. Only the gateway's enabled models are offered; a model the server no longer lists is shown but cannot be chosen.
3. Choose Save on that row. Each row is saved on its own and shows when it was last changed.

Changes apply from Sentinel's next run, and Sentinel runs every 30 minutes by default. A row can carry one warning:

How a service picks its gateway and model

At the start of every run, Sentinel asks the platform for a gateway once per stage, naming the stage's purpose — sentinel.triage, sentinel.brain or sentinel.code. The platform hands over the gateway's base URL, API key and enabled models:

• A stage with an assignment gets the assigned gateway and model.
• A stage without one gets the default gateway and uses its first enabled model, which can be a model marked Missing.

Nothing is kept between runs, so a new key, a changed default, a newly enabled model or a changed assignment takes effect on the next run without a restart.

When a stage cannot get what it needs — no gateway is registered, the default gateway has no enabled models, Sentinel lacks the permission, or the platform API cannot be reached — Sentinel skips the run and records why, and the next run tries again. A gateway that is not Healthy is still handed out.

Sentinel's older settings for its own endpoints, gateway and models — Agent__TriageBaseUrl, Agent__BrainBaseUrl, Agent__CodeBaseUrl, Agent__Gateway, Agent__ModelName and the per-stage Agent__TriageModel, Agent__BrainModel and Agent__CodeModel — are not set by the installer, and rolling Sentinel out removes them from its configuration. What is chosen on these tabs is what Sentinel uses.

Permissions

Every action is checked at the platform root, so only roles assigned there grant it; a role assigned on a boundary, Owner included, never does.

Platform Owner and Platform Contributor hold all four, resolve included. Platform Reader holds read and sees both tabs without their controls. The sidebar shows Platform Settings only to those who can change the e-mail settings, so a Platform Reader opens the page at https://portal.example.com/admin/platform-settings.

The LLM Gateway Consumer role holds resolve and none of the other three, and can only be assigned at the platform root. The platform's RBAC service grants it there to Sentinel's service account — the Keycloak client stackship-module-sentinel — and grants it again if it is removed; see Installation settings.

What is recorded

Every change is recorded in the platform's audit log under the resource type llmgateways, with the gateway's slug as the resource name:

The key itself is never recorded. A resolve that cannot be recorded is refused, so no key is handed out unrecorded; a change that cannot be recorded is still saved. Checks — Test connection, Re-verify and the periodic check — are not recorded, and neither are the health and model-list updates they make.

Installation settings

Set them as environment variables, for example LlmGateways__HealthCheckInterval=00:05:00 or ModuleGrantReconciler__LlmGatewayConsumerClientIds__0=stackship-module-sentinel. The RBAC module checks these grants when it starts and then every 10 minutes (ModuleGrantReconciler:SweepInterval), and grants again any that is missing. A client that does not exist is skipped.

With the API

The portal uses these routes under https://api.example.com. Every one is checked at the platform root.

POST https://api.example.com/admin/llm-gateways
Content-Type: application/json

{ "name": "LiteLLM", "slug": "litellm", "baseUrl": "https://litellm.example.com", "apiKey": "sk-…" }

In a PUT to a gateway every field is optional. apiKey left out or null keeps the stored key, "" removes it, and any other value replaces it. models, when given, is the complete list of { "id": …, "enabled": … } entries: manual models missing from it are removed, and models the server lists are kept. "isDefault": false on the default gateway leaves the platform with no default, so services that name no gateway get none — make another gateway the default instead.

An assignment body is { "gatewayId": "…", "model": "…" }; { "gatewayId": null } clears the assignment. Invalid input is answered with 400 and a message per field, and a conflict — a slug that is taken, deleting the default or an assigned gateway, disabling an assigned model — with 409.