AI Gateway Examples
This page gives copyable operating examples for soha and the AI Gateway. The examples assume the backend is reachable at http://localhost:8080, the bootstrap admin user can manage Gateway objects, and soha is available on the local PATH.
Do not paste real tokens, kubeconfig content, environment variables, or raw log bodies into tickets or AI conversations. The commands below use placeholders for all secret values.
Bootstrap Profiles
Create one admin profile for Gateway management:
soha login \
--server http://localhost:8080 \
--login admin \
--profile gateway-admin \
--ai-client soha-admin \
--ai-client-id soha-adminCreate local profiles for interactive clients. The AI client id and skill id become Gateway headers and are used by audit, tool grants, access policies, and skill bindings.
Register new team clients as pending when onboarding should require review:
curl -sS -X POST "$SOHA_SERVER/api/v1/ai-gateway/ai-clients" \
-H "Authorization: Bearer $SOHA_ADMIN_TOKEN" \
-H "Content-Type: application/json" \
-d '{
"id": "codex-local",
"name": "Codex Local",
"kind": "mcp_client",
"status": "pending_approval",
"metadata": {"owner":"platform"}
}'The create call writes a pending approval request. Approving it activates the client:
soha governance status --profile gateway-admin --json
curl -sS -X POST "$SOHA_SERVER/api/v1/ai-gateway/approval-requests/$APPROVAL_REQUEST_ID/approve" \
-H "Authorization: Bearer $SOHA_ADMIN_TOKEN" \
-H "Content-Type: application/json" \
-d '{"comment":"approved for local development"}'soha context set \
--profile gateway-admin \
--ai-client-id codex-local \
--ai-client Codex \
--skill-id delivery-developer \
--source sohaMCP Client Configuration
Generate the canonical JSON shape:
soha mcp install --profile gateway-admin --command /usr/local/bin/sohaCursor and Claude Code use the mcpServers JSON shape:
{
"mcpServers": {
"soha": {
"command": "/usr/local/bin/soha",
"args": ["mcp", "start", "--profile", "gateway-admin"]
}
}
}Codex uses TOML in ~/.codex/config.toml:
[mcp_servers.soha]
command = "/usr/local/bin/soha"
args = ["mcp", "start", "--profile", "gateway-admin"]For project-local or team-managed setups, keep the soha profile in the user account that runs the editor. The MCP process reads ~/.soha/config.json by default, or the path in SOHA_CONFIG.
CI Service Account
Create a service account for CI and give it only the roles or scope grants required by the pipeline:
soha service-account create \
--profile gateway-admin \
--id ci-delivery \
--name "CI Delivery" \
--role-ids delivery-operator \
--metadata-json '{"owner":"platform","purpose":"delivery-ci"}'Create a short-lived token. The token value is printed once; store it in the CI secret store, not in repository files.
soha service-account token-create \
--profile gateway-admin \
--service-account-id ci-delivery \
--name "github-actions" \
--permission-keys ai.gateway.view,ai.gateway.invoke \
--expires-at 2026-12-31T00:00:00ZCI can then use an isolated profile file:
export SOHA_CONFIG="$RUNNER_TEMP/soha-config.json"
export SOHA_TOKEN="$SOHA_GATEWAY_TOKEN"
export SOHA_SERVER="https://soha.example.com"
soha context set \
--profile ci \
--ai-client-id ci-github-actions \
--ai-client "GitHub Actions" \
--skill-id delivery-developer \
--source ci
soha capabilities --profile ci --output namesIf the CI job only needs MCP, point the client at:
soha mcp start --profile ci --ai-client-id ci-github-actions --ai-client "GitHub Actions" --skill-id delivery-developerLLM Relay Examples
These examples target the relay OpenAPI contract. As of 2026-06-25, the backend domain models, permission keys, token metadata, repository interface, and config defaults are present, while relay HTTP handlers and routes are still being completed in parallel. Run the SDK and curl calls only after the relay routes are registered; before that point, use the OpenAPI x-soha-implementation-status markers as the source of truth.
Create a relay-scoped PAT or SAT instead of reusing an old MCP-only token. The important boundary is both permission and metadata: ai.gateway.relay.invoke allows relay invocation, while metadata.purpose=llm-relay or metadata.scopes containing relay prevents legacy Gateway keys from becoming relay keys by accident.
curl -sS -X POST "$SOHA_SERVER/api/v1/ai-gateway/personal-access-tokens" \
-H "Authorization: Bearer $SOHA_ADMIN_TOKEN" \
-H "Content-Type: application/json" \
-d '{
"name": "local-relay-dev",
"scopes": ["ai_gateway", "relay"],
"permissionKeys": ["ai.gateway.relay.invoke"],
"metadata": {
"purpose": "llm-relay",
"scopes": ["relay"],
"allowedModels": ["gpt-4.1", "claude-sonnet-4-5"],
"allowedProviderKinds": ["openai", "anthropic"],
"allowedUpstreamIds": [],
"allowedIPCIDRs": ["10.0.0.0/8"]
},
"expiresAt": "2026-12-31T00:00:00Z"
}'Configure upstreams with environment variables or a secret manager. Do not paste provider keys into tickets, shell transcripts, docs, or AI conversations.
curl -sS -X POST "$SOHA_SERVER/api/v1/ai-gateway/relay/upstreams" \
-H "Authorization: Bearer $SOHA_ADMIN_TOKEN" \
-H "Content-Type: application/json" \
-d '{
"id": "openai-primary",
"name": "OpenAI primary",
"providerKind": "openai",
"baseUrl": "https://api.openai.com/v1",
"apiKey": "'"$OPENAI_UPSTREAM_API_KEY"'",
"status": "active",
"priority": 10,
"weight": 100,
"timeoutSeconds": 120,
"streamTimeoutSeconds": 300,
"maxConcurrency": 20,
"supportedModels": ["gpt-4.1"]
}'
curl -sS -X POST "$SOHA_SERVER/api/v1/ai-gateway/relay/model-routes" \
-H "Authorization: Bearer $SOHA_ADMIN_TOKEN" \
-H "Content-Type: application/json" \
-d '{
"id": "route-gpt-4-1",
"publicModel": "gpt-4.1",
"providerKind": "openai",
"upstreamId": "openai-primary",
"upstreamModel": "gpt-4.1",
"routeGroup": "default",
"priority": 10,
"weight": 100,
"enabled": true
}'OpenAI SDK with the Responses API:
import OpenAI from "openai";
const client = new OpenAI({
apiKey: process.env.SOHA_RELAY_TOKEN,
baseURL: `${process.env.SOHA_SERVER}/api/v1/ai-gateway/llm/openai/v1`,
});
const response = await client.responses.create({
model: "gpt-4.1",
input: "Return one sentence describing the relay smoke test.",
});
console.log(response.output_text);OpenAI SDK with Chat Completions streaming:
import OpenAI from "openai";
const client = new OpenAI({
apiKey: process.env.SOHA_RELAY_TOKEN,
baseURL: `${process.env.SOHA_SERVER}/api/v1/ai-gateway/llm/openai/v1`,
});
const stream = await client.chat.completions.create({
model: "gpt-4.1",
messages: [{ role: "user", content: "Stream three short words." }],
stream: true,
stream_options: { include_usage: true },
});
for await (const chunk of stream) {
process.stdout.write(chunk.choices[0]?.delta?.content ?? "");
}Anthropic SDK with Messages:
import Anthropic from "@anthropic-ai/sdk";
const client = new Anthropic({
apiKey: process.env.SOHA_RELAY_TOKEN,
baseURL: `${process.env.SOHA_SERVER}/api/v1/ai-gateway/llm/anthropic`,
});
const message = await client.messages.create({
model: "claude-sonnet-4-5",
max_tokens: 256,
messages: [{ role: "user", content: "Return one sentence describing the relay smoke test." }],
});
console.log(message.content);curl against native compatible endpoints:
curl -sS "$SOHA_SERVER/api/v1/ai-gateway/llm/openai/v1/models" \
-H "Authorization: Bearer $SOHA_RELAY_TOKEN"
curl -sS "$SOHA_SERVER/api/v1/ai-gateway/llm/openai/v1/responses" \
-H "Authorization: Bearer $SOHA_RELAY_TOKEN" \
-H "Content-Type: application/json" \
-d '{"model":"gpt-4.1","input":"Say ok."}'
curl -sS "$SOHA_SERVER/api/v1/ai-gateway/llm/anthropic/v1/messages" \
-H "x-api-key: $SOHA_RELAY_TOKEN" \
-H "anthropic-version: 2023-06-01" \
-H "Content-Type: application/json" \
-d '{"model":"claude-sonnet-4-5","max_tokens":64,"messages":[{"role":"user","content":"Say ok."}]}'Relay security checklist:
- Upstream keys are encrypted at rest and read APIs return only
apiKeyPrefix. - PAT/SAT plaintext values are shown once; logs, audits, model-call metadata, errors, and frontend state use token prefixes only.
ai.gateway.relay.invokeis required for native model traffic, and token metadata must explicitly identify relay purpose or scope.metadata.allowedModels,allowedProviderKinds,allowedUpstreamIds,allowedIPCIDRs, and rate-limit profile ids narrow relay use.- Model-call logs keep model, upstream, endpoint, status, latency, usage counts, cache-token counts, and redacted metadata; they do not store prompt bodies, full headers, Authorization values, provider keys, or raw provider payloads.
- Operators must configure only upstream accounts they are authorized to use and must follow provider and internal data-handling requirements.
P0 Relay Smoke Checklist
Use this checklist after the backend relay route worker lands the HTTP handlers:
- OpenAPI parses and contains relay upstream, model-route, model-call, metrics, cache, OpenAI-compatible, and Anthropic-compatible paths.
- A relay PAT or SAT with
ai.gateway.relay.invokeandmetadata.purpose=llm-relaycan call OpenAI SDK non-streaming and streaming requests through the Soha base URL. - The same relay token can call Anthropic SDK non-streaming and streaming Messages requests through the Soha base URL.
- A token without
ai.gateway.relay.invokeor relay metadata receives403. - Disabled, expired, or revoked PAT/SAT credentials cannot invoke relay endpoints.
- Upstream API keys do not appear in Console/API read responses, model-call logs, audit logs, application logs, or error responses.
GET /api/v1/ai-gateway/relay/model-callsshows model, upstream, status, TTFB, duration, prompt/completion/total tokens, and cache-token fields after successful calls.- Streaming client disconnect cancels the upstream request and records
client_cancelled. - Updating an upstream or model route invalidates relay config cache before the next routed request.
- Existing MCP
POST /api/v1/ai-gateway/tools/:toolName/invokebehavior still works with a normal Gateway token.
Console Field Mapping
The Console management page is /ai-gateway. It is the operator surface for the same Gateway API objects used by soha and MCP. ai.gateway.view can inspect the page, ai.gateway.invoke can create personal tokens, and ai.gateway.manage is required for AI clients, service accounts, grants, policies, bindings, and approval decisions.
Use this mapping when translating an operational change request into Console input fields:
- Manifest maps to
GET /api/v1/ai-gateway/capabilities. Filters areAI client,Skill, andsource. The tool table is a preview of visible capabilities only; every invocation still re-checks permission keys, scopes, grants, access policies, skill bindings, and owning-service authorization. - AI clients maps to
ai_clients. The form fields areClient ID,name,kind,status,redirectUris, andallowedOrigins. Console creates and edits active or disabled clients; registration-review onboarding can still use the API path withpendingorpending_approval, then approve it from the approval tab. - Personal access tokens maps to
personal_access_tokens. The create form fields arename,permissionKeys,scopes, andexpiresAt. The plaintext token is shown once after creation or rotation and is not stored in browser state; rotation copies the old token boundary and revokes the old token. - Service accounts maps to
service_accountsandservice_account_tokens. Service account fields are optionalid,name,description,status,roleIds,teamIds, andscopeGrantIds. Service token fields arename,permissionKeys,scopes, andexpiresAt; the token table shows id, owner service account, prefix, permissions, scopes, expiration, last-used, revocation/expiration state, rotate, and revoke actions. - MCP tool grants maps to
mcp_tool_grants. The form fields aresubjectType,subjectId, optionalaiClientId,toolName,effect, optionalriskLevel, extrapermissionKeys,requiresApproval, and resource scope fields for business line, application, application environment, environment, cluster, namespace, release bundle, and execution task. - Access policies maps to
ai_access_policies. Console covers the policy envelope:name,description,enabled,subjectType,subjectId, optionalaiClientId,effect,toolPatterns,skillIds,riskLevels,approvalMode, and the same resource scope fields. It also provides structured controls for common approval routing (approvalPolicyRef, candidate users/roles/teams,onCallRef,approvalMode=all|any,requiredApprovals, and change-window bounds) plus common governance conditions:rateLimitfixed-window, sliding-window, or GCRA/token-bucket limits, daily invocation/token/costbudget, inputredactionPolicywith field, regex, and secret-classifier selectors, andoutputRedactionPolicywith output field, regex, secret-classifier, replacement, and preserve-format controls. Staged approval and per-tool redactionrules[]remain API/cookbook fields. - Skill bindings maps to
ai_gateway_skill_bindings. The form fields aresubjectType,subjectId, optionalaiClientId,skillId,capabilityRefs, andenabled. Bindings narrow the manifest for a skill; they do not grant business permissions. - Governance maps to
GET /api/v1/ai-gateway/governance/status. The window selector sendswindowHoursvalues for 1h, 6h, 24h, 48h, or 7d. The tab shows health status and checks, recent calls, token/client/approval summaries, policy coverage rows, redaction hit summary, token findings, approval queues, top tools, top AI clients, top actors, anomaly findings, string recommendations, and structuredrecommendationActions. The redaction hit summary is aggregated from Gateway audit metadata and includes total matches, redacted audit count, input/output targets, match types, classifiers, field paths, policies, and top tools. Token findings can filter the Personal access token table, filter the service-account token table, or prefill service-token revocation by token id; redaction policy/tool rows can jump to the matching policy or audit filter view; approval queues, anomaly findings, and recommendation actions can jump to the matching approval, AI client, policy, grant, or audit filter view. Coverage gaps and high-risk findings/actions can open a prefilled access policy draft for budget, rate-limit, redaction, resource-scope, or approval guardrails. Policy coverage treats active inputredactionPolicy,outputRedactionPolicy, and response/output redaction aliases as redaction coverage. It requiresai.gateway.manageand only exposes redacted summaries, request ids, and token prefixes. - Approval requests maps to
ai_gateway_approval_requests. Filters areapprovalRequestId,status,actorId,AI client,Tool,Risk,Strategy, and created-time range. Rows expose status, actor, client/skill, tool, risk, strategy, expiry, summary, and a trace column. The expanded trace shows Gateway approval id, workflow run link, application, application environment, execution task, release bundle, decision comment, resource scope, redacted tool input, related ids, and output. Approve, reject, and cancel actions accept a comment. - Audit maps to
ai_gateway_audit_logs. Filters areactorId,AI client,Tool,Risk,Result, and created-time range. Expanded rows show only resource scope, request id, and source IP; raw tool input, token values, kubeconfig, environment variables, and raw log bodies are not exposed.
Runtime configuration is not edited from this page. For example, the Redis rate-limit backend is configured with ai_gateway.rate_limit.backend=redis and ai_gateway.rate_limit.redis.addr or the matching environment variables, while policies in Console or API decide which invocations use rate limits.
Delivery Developer Flow
Use the delivery-developer skill for self-service application delivery work. The recommended flow is read context first, then request a controlled action.
Inspect the application and environment context:
soha tool call delivery.applications.detail \
--profile gateway-admin \
--input-json '{"applicationId":"app-1"}'
soha tool call delivery.application_services.list \
--profile gateway-admin \
--input-json '{"applicationId":"app-1"}'
soha tool call delivery.build_sources.list \
--profile gateway-admin \
--input-json '{"applicationId":"app-1","includeBindings":true}'Inspect release targets and current execution state:
soha tool call delivery.release_targets.list \
--profile gateway-admin \
--input-json '{"applicationEnvironmentId":"binding-1"}'
soha tool call delivery.release_context.diff \
--profile gateway-admin \
--input-json '{"applicationId":"app-1","sourceReleaseBundleId":"bundle-old","targetReleaseBundleId":"bundle-new"}'Trigger a controlled build or deploy action. Mutating tools still go through access policy risk strategy and the owning delivery application service.
soha tool call delivery.actions.trigger \
--profile gateway-admin \
--input-json '{"applicationId":"app-1","applicationEnvironmentId":"binding-1","action":"build_deploy"}'For rollback, collect read-only context first, then trigger the controlled rollback action. The bound workflow template must include a rollback_to_previous DAG node; Gateway approval replay keeps releaseBundleId and workflowRunId linked.
soha tool call delivery.rollback.context \
--profile gateway-admin \
--input-json '{"applicationId":"app-1","applicationEnvironmentId":"binding-1","releaseBundleId":"bundle-prev"}'
soha tool call delivery.actions.trigger \
--profile gateway-admin \
--input-json '{"applicationId":"app-1","applicationEnvironmentId":"binding-1","action":"rollback","releaseBundleId":"bundle-prev"}'If the matching policy requires approval, the result is pending_approval or pending_human_confirm with an approval request id. The caller should not retry by bypassing Gateway.
K8s SRE Release Failure Diagnosis
Use k8s-sre for bounded runtime diagnosis. Start with rollout and event context:
soha tool call k8s.deployments.rollout_status \
--profile gateway-admin \
--input-json '{"clusterId":"cluster-a","namespace":"prod","deploymentName":"api"}'
soha tool call k8s.deployments.events \
--profile gateway-admin \
--input-json '{"clusterId":"cluster-a","namespace":"prod","deploymentName":"api","limit":50}'Inspect pod and service context:
soha tool call k8s.pods.describe \
--profile gateway-admin \
--input-json '{"clusterId":"cluster-a","namespace":"prod","podName":"api-7d9f"}'
soha tool call k8s.services.backends \
--profile gateway-admin \
--input-json '{"clusterId":"cluster-a","namespace":"prod","serviceName":"api"}'Run the release-failure analyzer. The Gateway stores a redacted AnalysisArtifact through the Copilot application service and does not persist raw log bodies. In AI Workbench, artifact history exposes any stored sessionId, rootCauseRunId, inspectionRunId, and agentRunId / agentRuntimeId as context links; inspection-review artifacts can open the inspection runs view directly with view=runs&inspectionRunId=....
soha tool call diagnosis.release_failure.analyze \
--profile gateway-admin \
--input-json '{"applicationId":"app-1","applicationEnvironmentId":"binding-1","releaseBundleId":"bundle-new","executionTaskId":"task-1","clusterId":"cluster-a","namespace":"prod","workloadName":"api","podName":"api-7d9f"}'For deeper reasoning through an external Agent Runtime provider, request asynchronous analysis. Gateway still only collects and redacts context; it creates a queued AgentRun and returns agentRunId, agentProviderId, and agentRunStatus. A configured agent runner must later claim the run and callback with normalized analysisArtifacts.
soha tool call diagnosis.release_failure.analyze \
--profile gateway-admin \
--input-json '{"applicationId":"app-1","applicationEnvironmentId":"binding-1","releaseBundleId":"bundle-new","executionTaskId":"task-1","clusterId":"cluster-a","namespace":"prod","workloadName":"api","podName":"api-7d9f","deepAnalysis":true,"agentProviderId":"hermes","timeoutSeconds":900}'Access Policy Cookbook
Create policies with POST /api/v1/ai-gateway/access-policies. Handlers stay thin; runtime enforcement happens in the AI Gateway application service before any owning application service is called.
Read-only delivery policy for local Codex:
curl -sS -X POST "$SOHA_SERVER/api/v1/ai-gateway/access-policies" \
-H "Authorization: Bearer $SOHA_ADMIN_TOKEN" \
-H "Content-Type: application/json" \
-d '{
"name": "Codex read-only delivery",
"enabled": true,
"subjectType": "role",
"subjectId": "developer",
"aiClientId": "codex-local",
"effect": "allow",
"toolPatterns": ["delivery.*.list", "delivery.applications.detail", "delivery.release_context.diff"],
"riskLevels": ["read", "analyze"],
"conditions": {
"rateLimit": {"maxCallsPerMinute": 60, "scope": "actor_client", "mode": "gcra", "burst": 20},
"budget": {"maxCallsPerDay": 500, "scope": "actor_client"},
"redactionPolicy": {"mode": "sanitize"}
}
}'Require approval for delivery mutations:
curl -sS -X POST "$SOHA_SERVER/api/v1/ai-gateway/access-policies" \
-H "Authorization: Bearer $SOHA_ADMIN_TOKEN" \
-H "Content-Type: application/json" \
-d '{
"name": "Delivery mutation approval",
"enabled": true,
"subjectType": "role",
"subjectId": "developer",
"aiClientId": "codex-local",
"effect": "allow",
"toolPatterns": ["delivery.actions.trigger"],
"riskLevels": ["execute", "high"],
"approvalPolicy": {
"strategy": "require_approval",
"approvalPolicyRef": "delivery-standard",
"approvalMode": "all",
"approverRoles": ["release-manager"],
"approverTeams": ["platform-ops"],
"requiredApprovals": 2,
"requiredRoleApprovals": {"release-manager": 1},
"requiredTeamApprovals": {"platform-ops": 1},
"approvalStages": [
{
"name": "release",
"approverRoles": ["release-manager"],
"requiredApprovals": 1
},
{
"name": "security",
"approverRoles": ["security-reviewer"],
"requiredApprovals": 1
}
],
"onCallRef": "prod-release-oncall",
"changeWindow": {
"startsAt": "2026-05-29T10:00:00Z",
"endsAt": "2026-05-29T12:00:00Z",
"timezone": "UTC"
}
},
"conditions": {
"rateLimit": {"maxCallsPerHour": 10, "scope": "actor_client_tool"},
"redactionPolicy": {"mode": "strict"}
}
}'Deny a shared client from high-risk tools:
curl -sS -X POST "$SOHA_SERVER/api/v1/ai-gateway/access-policies" \
-H "Authorization: Bearer $SOHA_ADMIN_TOKEN" \
-H "Content-Type: application/json" \
-d '{
"name": "Shared client high-risk deny",
"enabled": true,
"subjectType": "ai_client",
"subjectId": "shared-agent",
"effect": "deny",
"riskLevels": ["high", "execute", "mutate"]
}'Scope an SRE policy to one cluster and namespace:
curl -sS -X POST "$SOHA_SERVER/api/v1/ai-gateway/access-policies" \
-H "Authorization: Bearer $SOHA_ADMIN_TOKEN" \
-H "Content-Type: application/json" \
-d '{
"name": "SRE prod runtime read",
"enabled": true,
"subjectType": "role",
"subjectId": "sre",
"effect": "allow",
"toolPatterns": ["k8s.*", "diagnosis.release_failure.analyze"],
"riskLevels": ["read", "analyze"],
"resourceScopes": {"clusterId": "cluster-a", "namespace": "prod"},
"conditions": {
"rateLimit": {"maxCallsPerMinute": 120, "scope": "actor_client", "mode": "token_bucket", "burst": 30},
"budget": {"maxCallsPerDay": 1000, "scope": "actor_client"}
}
}'Condition notes:
rateLimitdefaults to the Gateway fixed-window counter table for atomic enforcement. Setmode/algorithm/strategytosliding_window,rolling_window, oraudit_windowto use the Gateway audit log as a true rolling window. Set it togcra,token_bucket,leaky_bucket, orstrictto use GCRA/TAT semantics, withburst/burstSize/capacitycontrolling burst capacity. By default this uses PostgreSQL-backed rate-limit tables; setai_gateway.rate_limit.backend=redisplusai_gateway.rate_limit.redis.addrto use Redis INCR/EXPIRE and Lua GCRA as the high-throughput shared limiter. Redis failures fall back to PostgreSQL, and older databases without the matching migration still temporarily fall back to the audit-window count.- Docker Compose can start the optional Redis limiter with
docker compose --profile redis-rate-limit up; setSOHA_AI_GATEWAY_RATE_LIMIT_BACKEND=redisand leaveSOHA_AI_GATEWAY_RATE_LIMIT_REDIS_ADDR=redis:6379or point it at an external Redis endpoint. approvalPolicy.approvalPolicyRefrecords the Gateway approval policy reference. Gateway approval timeout and routing come from the same inlineapprovalPolicyobject, includingslaMinutes,approvalMode, required approval counts, approver users/roles/teams, change windows, and staged routing.approvalPolicy.approverUsers,approverRoles,approverTeams,onCallRef,approvalMode,approvalStages,requiredApprovals/minApprovals,requiredRoleApprovals/requiredTeamApprovals, andchangeWindow.startsAt/endsAtare persisted as approval routing metadata.onCallRefis resolved through the monitoring on-call resolver; the current participant is appended tocandidateUserIds, and Gateway storesonCallCandidateUserIdsplus a redactedonCallResolutionsummary. Decision APIs reject non-candidate approvers; approve also requires the current time to be inside the change window. Multi-approval quorum counts unique approvinguserIds and replaces the same approver’s prior decision instead of double-counting it.approvalMode=allis the default joint approval mode and keeps the requestpendinguntil the total count and all role/team quotas are reached;approvalMode=anyis the alternate approval mode and executes when any explicit total, role, or team quota is satisfied. Pending requests recordpendingRequirements; satisfied paths recordsatisfiedRequirements. Staged approval usesapprovalStages[]; only the current stage candidates can approve, stage completion advancescurrentStageIndexand recordsstageHistory, and only the final stage replays the tool.- When an approved Gateway request replays
delivery.actions.trigger, the replay injectsaiGatewayApprovalRequestIdand related policy/tool/skill/client fields into delivery workflow variables. Workflow run metadata keeps those values, workflowmanual_approvaldecisions persist them inworkflow_approvals.metadata, and Gateway/delivery related IDs includeworkflowRunIdfor traceability. Opening/workflows?workflowRunId=...&gatewayApprovalRequestId=...auto-expands the workflow row with Gateway linkage, manual approval node detail, workflow node timeline, and raw trace. budget.maxCalls*is an invocation-count budget.budget.maxTokens*andbudget.maxCost*are usage budgets based on redacted Gateway audit metadata such asusage,tokenUsage, orproviderUsage.- On successful tool calls and approval replay, Gateway extracts numeric token/cost summaries from owning-service output or related IDs when they contain
usage,tokenUsage,aiUsage,providerUsage,llmUsage,metering,billing, orcostUsage. It also maps native fields such as Gemini/VertexusageMetadata.promptTokenCount/candidatesTokenCount/totalTokenCount, cached content, tool-use prompt, and thoughts token fields, Ollamaprompt_eval_count/eval_count, Anthropic cache input tokens, OpenAI cached/audio/reasoning/prediction token details, Bedrock text/image/audio token fields, Cohere billed units with same-containerbilled_unitspreferred overtokensto avoid double counting, search/classification/embedding/rerank/request/response usage units, Brave Search / SerpAPI query and search-credit aliases, Browserbase session/minute/page-load aliases, Exa/Jina/Unstructured/LlamaCloud document-page/parse-page/character/chunk aliases, Helicone request-count aliases, billable/billed/usage token aliases, multimodal text/image/audio/video input/output aliases, cache read/write/hit/miss input token variants, DashScope/Moonshot/Zhipu/Qianfan-style*_tokens_count/*_token_usagevariants, genericpromptCost/inputCost/completionCost/outputCost, and common cost / costMicros / costCents fields. It writes only numericproviderUsage/usagesummaries to audit metadata; raw provider payloads, model strings, and other non-numeric details are not stored. redactionPolicy.mode=strictrejects sensitive keys such astoken,password,secret,authorization,kubeconfig, and secret-like text before the tool reaches the owning service.redactionPolicy.mode=sanitize|mask|redactrewrites sensitive input values to[REDACTED]before the owning service sees them.redactionPolicy.fields/redactFieldsnarrows replacement to field paths.allowFieldsexempts trusted field paths.replacementcustomizes the replacement string.preserveFormat=truemasks a value while keeping the last four characters.redactionPolicy.valuePatternsadds custom regex-based value matching.secretTypesenables built-in classifiers such asgithub,gitlab,openai,anthropic,google_api_key,huggingface,cohere,mistral,deepseek,groq,together,replicate,langsmith,pinecone,xai,perplexity,tavily,langfuse,qdrant,wandb,linear,openrouter,fireworks,voyage,brave_search,serpapi,browserbase,exa,jina,unstructured,llama_cloud,helicone,dashscope,moonshot,zhipu,siliconflow,hunyuan,qianfan,volcengine,grafana,sentry,newrelic,azure_openai,azure_devops,datadog,pagerduty,posthog,splunk,elastic,terraform,npm,stripe,slack,jwt,aws,private_key,kubernetes_secret,kubeconfig,docker_config,gcp_service_account, andaws_credentials.- Redaction hits are written to Gateway audit metadata as a
redactionsummary with target, field paths, match types, classifier names, and counts only; the matched secret value is not stored. Governance status aggregates the same metadata into an operating summary so operators can see where redaction is happening without inspecting raw tool input or output. redactionPolicy.rules[]can bind field rules totoolPatterns, so one policy can sanitize different inputs for delivery, k8s, and diagnosis tools.- Input redaction is the default. Use
outputRedactionPolicy, or set a ruletargettooutput/both, to redact owning-service output before Gateway returns or persists it. scopecan beactor,client,actor_client,actor_tool,client_tool,actor_client_tool, orglobal.
Token and cost budget example:
curl -sS -X POST "$SOHA_SERVER/api/v1/ai-gateway/access-policies" \
-H "Authorization: Bearer $SOHA_ADMIN_TOKEN" \
-H "Content-Type: application/json" \
-d '{
"name": "Codex daily usage budget",
"enabled": true,
"subjectType": "ai_client",
"subjectId": "codex-local",
"effect": "allow",
"toolPatterns": ["delivery.*", "k8s.*", "diagnosis.*"],
"conditions": {
"budget": {
"maxTokensPerDay": 200000,
"maxCostPerDay": 15.0,
"scope": "actor_client"
}
}
}'Field-level redaction with an allow-list and format-preserving mask:
curl -sS -X POST "$SOHA_SERVER/api/v1/ai-gateway/access-policies" \
-H "Authorization: Bearer $SOHA_ADMIN_TOKEN" \
-H "Content-Type: application/json" \
-d '{
"name": "Delivery field redaction",
"enabled": true,
"subjectType": "role",
"subjectId": "developer",
"effect": "allow",
"toolPatterns": ["delivery.*"],
"riskLevels": ["read", "analyze", "mutate"],
"conditions": {
"redactionPolicy": {
"mode": "mask",
"fields": ["metadata.apiToken", "buildSources.*.config.password"],
"allowFields": ["search"],
"preserveFormat": true,
"rules": [
{
"toolPatterns": ["delivery.applications.create"],
"fields": ["metadata.apiToken"],
"replacement": "[MASKED]",
"preserveFormat": true
},
{
"toolPatterns": ["diagnosis.*"],
"fields": ["context.rawLogs", "context.env"],
"mode": "redact"
}
]
}
}
}'Regex and structured secret classification:
curl -sS -X POST "$SOHA_SERVER/api/v1/ai-gateway/access-policies" \
-H "Authorization: Bearer $SOHA_ADMIN_TOKEN" \
-H "Content-Type: application/json" \
-d '{
"name": "Secret classifier redaction",
"enabled": true,
"subjectType": "role",
"subjectId": "developer",
"effect": "allow",
"toolPatterns": ["delivery.*", "k8s.*", "diagnosis.*"],
"conditions": {
"redactionPolicy": {
"mode": "sanitize",
"valuePatterns": ["APP-[0-9]{4}", "ticket-[A-F0-9]{8}"],
"secretTypes": ["github", "openai", "anthropic", "google_api_key", "huggingface", "cohere", "mistral", "deepseek", "groq", "together", "replicate", "langsmith", "pinecone", "xai", "perplexity", "tavily", "langfuse", "qdrant", "wandb", "linear", "openrouter", "fireworks", "voyage", "brave_search", "serpapi", "browserbase", "exa", "jina", "unstructured", "llama_cloud", "helicone", "dashscope", "moonshot", "zhipu", "siliconflow", "hunyuan", "qianfan", "volcengine", "grafana", "sentry", "newrelic", "azure_openai", "azure_devops", "datadog", "pagerduty", "posthog", "splunk", "elastic", "terraform", "npm", "stripe", "jwt", "aws", "private_key", "kubernetes_secret", "kubeconfig", "docker_config", "gcp_service_account"],
"replacement": "[CLASSIFIED]",
"preserveFormat": false
}
}
}'Output redaction for tool responses and approval replay:
curl -sS -X POST "$SOHA_SERVER/api/v1/ai-gateway/access-policies" \
-H "Authorization: Bearer $SOHA_ADMIN_TOKEN" \
-H "Content-Type: application/json" \
-d '{
"name": "Delivery output redaction",
"enabled": true,
"subjectType": "role",
"subjectId": "developer",
"effect": "allow",
"toolPatterns": ["delivery.*"],
"riskLevels": ["read", "analyze", "mutate", "execute"],
"conditions": {
"outputRedactionPolicy": {
"mode": "sanitize",
"fields": [
"application.buildSources.*.config.token",
"build.metadata.token",
"release.metadata.secret"
]
}
}
}'Skill Binding Cookbook
Skill bindings narrow which capabilities a subject or AI client can use under a skill. They do not grant business permissions by themselves.
Bind Codex to the delivery developer skill:
curl -sS -X POST "$SOHA_SERVER/api/v1/ai-gateway/skill-bindings" \
-H "Authorization: Bearer $SOHA_ADMIN_TOKEN" \
-H "Content-Type: application/json" \
-d '{
"subjectType": "ai_client",
"subjectId": "codex-local",
"skillId": "delivery-developer",
"enabled": true,
"capabilityRefs": [
"delivery.applications.*",
"delivery.application_services.list",
"delivery.build_sources.list",
"delivery.release_targets.list",
"delivery.release_bundles.list",
"delivery.execution_tasks.list",
"delivery.execution_logs.list",
"delivery.release_context.diff",
"delivery.rollback.context",
"delivery.actions.trigger"
],
"metadata": {"owner":"platform"}
}'Bind the SRE role to bounded runtime diagnosis:
curl -sS -X POST "$SOHA_SERVER/api/v1/ai-gateway/skill-bindings" \
-H "Authorization: Bearer $SOHA_ADMIN_TOKEN" \
-H "Content-Type: application/json" \
-d '{
"subjectType": "role",
"subjectId": "sre",
"skillId": "k8s-sre",
"enabled": true,
"capabilityRefs": [
"k8s.pods.*",
"k8s.deployments.*",
"k8s.services.backends",
"k8s.routes.context",
"k8s.storage.context",
"k8s.nodes.detail",
"diagnosis.release_failure.analyze"
]
}'Use diagnose to inspect what the current profile can see:
soha diagnose --profile gateway-admin --tool diagnosis.release_failure.analyze
soha capabilities --profile gateway-admin --jsonWhen an MCP client can list capabilities but resource or prompt context looks wrong, compare the backend-proxied resource and prompt paths directly:
soha resource read soha://delivery/applications \
--profile gateway-admin \
--skill-id delivery-developer \
--context-json '{"applicationId":"app-1"}'
soha prompt get soha.delivery.plan_release \
--profile gateway-admin \
--skill-id delivery-developer \
--arguments-json '{"applicationId":"app-1","applicationEnvironmentId":"dev"}' \
--context-json '{"environmentId":"dev"}'
soha diagnose --profile gateway-admin --resource soha://k8s/runtime
soha diagnose --profile gateway-admin --prompt soha.k8s.diagnose_workloadThese commands still proxy through Gateway resources/read and prompts/get, so failures should be debugged with the same runtime permissions, skill bindings, AI client context, and resource scopes used by the MCP server.
Operational Checks
Review audit and governance after onboarding a client:
soha audit list --profile gateway-admin --ai-client-id codex-local --limit 20
soha governance status --profile gateway-admin --window-hours 24Governance status should be clean before widening access: warnings for approval_sla_due_soon or stale_gateway_approvals mean pending Gateway approvals need operator attention before their SLA or queue age becomes a release blocker. Warnings for high_risk_allow_without_approval or high_risk_grant_without_approval mean an active allow policy or unexpired grant can reach high-risk tools without an approval, human-confirmation, or dry-run guard. Warnings for high_risk_allow_without_resource_scope or high_risk_grant_without_resource_scope mean the high-risk allow path has no concrete resourceScopes constraint, or only a wildcard scope such as "*".
If an expected tool is missing, check in this order:
- runtime permission keys such as
ai.gateway.invokeand the owning business permission; mcp_tool_grantsdeny/allow matching;ai_access_policiesdeny/allow, resource scope, risk strategy, and conditions;ai_gateway_skill_bindingsfor the currentskillId;- resource scope fields in the tool input, especially
clusterId,namespace,applicationId, andapplicationEnvironmentId.