Configuration
Source Of Truth
Startup configuration remains file-baselined; product settings and connection instances are persisted by the database-backed settings center.
- The source template is
configs/config.example.yaml. Compose mounts it at/app/configs/config.yaml. Local runs can use a privateconfigs/config.yamlor setSOHA_CONFIG_FILE. - loader:
internal/infrastructure/config - override mechanism: environment variables through Viper when needed
Frontend runtime is currently code-first rather than env-first.
- Web defaults to same-origin
/api/v1; the candidate prefix build described below derives<VITE_BASE_PATH>api/v1, unlessVITE_API_BASE_URLexplicitly overrides it - local development proxies
/apitohttp://127.0.0.1:8080in../soha/web/vite.config.ts - the docs surface inside the console opens
/docs/, which thesohaserver redirects to the configured external docs URL by default - there are no checked-in
../soha/web/.env.*templates at the moment
Docs runtime is independent.
- docs source lives in
soha-docs - published docs should use an external URL, such as
https://docs.opensoha.dev/ assets.docs.external_urlcontrols wheresoharedirects/docs/
Configuration Ownership
config.yaml is not the write surface for every product setting. Configuration is split by lifecycle and scope:
| Layer | Owns | Typical entry point |
|---|---|---|
| Deployment/startup | Listener, database, process topology, JWT/runner/webhook/encryption keys, bootstrap settings | config.yaml, environment variables, Docker/Kubernetes/Helm Secrets |
| Settings center | Login providers, the current global code-source integration, login policy, and safely reloadable product policy | Login settings, Code Sources, and Runtime Configuration pages |
| Cluster settings | kubeconfig/Agent connection plus that cluster’s Prometheus, Grafana, and resource-query parameters | Cluster detail/edit page |
| Runtime configuration | Versioned Soha module switches and runtime parameters applied immediately, by reconfiguration, or after restart | Settings-center Runtime Configuration page |
The Runtime Configuration page shows effective values; it is not an editor for every configuration value. Externally managed or cluster-scoped values should show their owner and a link to the owning page instead of creating a second write path.
Workflow workers, queue sizing, node parallelism, cluster synchronization, AI inspection, alert batching, virtualization workers, execution-job defaults, MCP timeout, and AI Gateway relay limits are persisted through Runtime Configuration and no longer need entries in config.yaml.
Code-source integration
The settings center currently exposes the system-integration backend through the Code Sources pages:
- The list and detail pages currently manage GitLab connections only. Identity providers remain under Login Settings, while alert integrations remain in the Monitoring Workbench.
- A GitLab detail page owns the endpoint, write-only credential, enablement, and connection test.
- GitLab is a global code source, not a delivery-workbench-only setting. Delivery, virtualization, container-runtime, and other features that read Dockerfiles, YAML, Helm charts, or scripts use the same connection.
- Workbenches store only a reference to the Provider (for example provider ID, project/repository, branch, and path); they do not copy GitLab tokens.
Integration credentials must be encrypted at rest and support connection testing. They are not ordinary flat runtime keys, and the browser must not call third-party systems directly.
Backend Config Shape
app:
http:
logger:
runtime:
database:
auth:
swagger:
mcp:
bootstrap:
kubernetes:
monitoring:
security:Key backend fields now used by the runtime:
http.cors_allowed_origins: allowed browser origins for the frontend consoleauth.jwt.secret,auth.jwt.access_ttl,auth.jwt.refresh_ttlauth.dev_principal.*: bootstrap local account seed and, when enabled, the no-token development principalauth.oidc.*: legacy runtime OIDC fallback only; it does not create settings-center login sourcesruntime.execution_runner_token: shared bearer token for delivery, Docker, and AI Agent Runtime runner claim/callback APIsdatabase.migration_path: migration root directory (runtime resolvesmigrations/<driver>/0001_init.sql)database.migration_file: explicit SQL bootstrap file override; the current repo baseline points at the consolidated PostgreSQL bootstrap filekubernetes.clusters[*].kubeconfig: direct cluster file path for bootstrapped clusterskubernetes.clusters[*].kubeconfig_data: inline kubeconfig support when file paths are not usedmcp.default_timeout: default timeout reused by MCP and agent HTTP integrationsmonitoring.enabled: toggles monitoring ingress endpointsmonitoring.webhook_token: shared token accepted by the alert webhook endpointsecurity.credential_encryption_key: key used to encrypt persisted integration and platform credentialssecurity.secret_provider: empty for local encrypted values, orvault_kv2to enable Vault-backed Secret versionssecurity.vault_kv2.*: server-side Vault address, token, namespace, timeout, and response-size limit
Vault KV v2 Secret Store
Vault KV v2 is the first external backend for Secret Store versions. Configure it only on the Soha server:
security:
secret_provider: vault_kv2
vault_kv2:
address: https://vault.example.com
token: replace-with-a-server-token
namespace: ""
timeout: 10s
max_response_bytes: 2097152The address must be an origin without credentials, path, query, or fragment. HTTPS is required except for loopback HTTP during local development. The token is never accepted by the public Secret API and is not stored in Secret records.
When creating or rotating a Vault-backed version, provide a KV v2 mount, path, data key, and an explicit positive Vault version. Soha stores only this locator. Callers still use soha://secrets/{id} or soha://secrets/{id}/versions/{version}; Core performs RBAC, scope and binding checks, approval, audit, and lease handling before reading Vault. Missing configuration or any Vault read failure remains fail-closed and does not expose the provider response.
Standard Initial Defaults
PostgreSQL
- version: PostgreSQL 18.4 with pgvector 0.8.5
- host:
localhost - port:
5432 - database:
soha - user:
pgsql - password:
pgsql
Identity Defaults
The standard bootstrap account comes from auth.dev_principal inside config.yaml.
- username:
opensoha - email:
opensoha@soha.local - password:
opensoha
When auth.enable_dev_auth is false, this account is still seeded into PostgreSQL for real password login. The flag only controls whether the backend accepts an automatic development principal when no bearer token is present. Routine restarts preserve the initial password without overwriting a user-changed password. Soha does not apply an environment-specific credential policy; explicit overrides remain supported.
System Secret Defaults
The project configuration keeps the four system secrets visible and gives all four the same standard default:
auth:
jwt:
secret: soha-123456789012345678901234567890
runtime:
execution_runner_token: soha-123456789012345678901234567890
monitoring:
webhook_token: soha-123456789012345678901234567890
security:
credential_encryption_key: soha-123456789012345678901234567890These defaults let a local process, docker run, Docker Compose, Kubernetes, and Helm start without a generated SecretStore file, writer lock, or dedicated secret PVC. The backend does not reject the documented value or stop startup because the four fields reuse it. Each field can still be overridden independently in config.yaml or through its environment override.
This is a convenience default for an open-source installation, not a production secret. Any Internet-facing or formal deployment should replace all four values before it accepts traffic. Leaving the public value unchanged allows third parties to forge JWTs and runner/webhook requests and to decrypt credentials obtained from the database.
Treat security.credential_encryption_key differently from bearer and signing secrets during a change: migrate or re-encrypt every existing ciphertext with the old key before removing it. Replacing this value without a migration makes existing encrypted credentials unreadable. Keep the configured encryption key stable across application restarts and all replicas until that migration is complete.
Login Provider Runtime
The active console login source is now settings-driven rather than file-only.
Operationally:
- settings-center login sources are stored only under
identity.login_providers auth.oidc.*does not seed or backfill login sources- operators can configure multiple concurrent providers from the console login-settings page
Supported provider types in the current settings model:
oidcfeishudingtalkwecomoauth2saml
Current runtime expectations:
oidc: full runtime supportedoauth2: supported when authorize/token/userinfo URLs and field mappings are correctfeishu: supported through configured Feishu auth endpoints and field mappingsdingtalk: supported through configured DingTalk auth endpoints and field mappingswecom: supported through webpage authorization plus enterprise access-token andgetuserinfoAPIssaml: supports SP metadata, ACS, signature and assertion validation, attribute mapping, and replay protection
Set http.access_url to the origin clients actually use. Both HTTP and HTTPS are supported for self-hosted installations; HTTPS is strongly recommended because identity tokens, assertions, and session cookies otherwise cross the network without transport protection. OIDC/SAML issuer and protocol URLs use this value instead of deriving authority from client-supplied forwarded host/proto headers.
Operational Conventions
- request ID header:
X-Request-Id - health check:
GET /healthz - readiness check:
GET /readyz - versioned API base path:
/api/v1 - swagger route reservation:
/swagger/*any - config override file:
SOHA_CONFIG_FILE=/abs/path/to/config.yaml - frontend local dev port:
5173 - docs local dev port:
3000
Database Bootstrap
On startup the backend can:
- run SQL migration file
- create schema only from the migration file
- seed bootstrap user from
auth.dev_principal - seed password hash and role bindings for the bootstrap account
- seed default RBAC roles
- seed default ABAC policies
- seed configured clusters into
clusters - seed file-configured direct cluster metadata into
cluster_credentials_meta
Cluster Registration API
The runtime now supports two persisted cluster connection modes:
direct_kubeconfig- request payload includes
kubeconfigand optionalcontext - kubeconfig is validated before the cluster is registered into
cluster-manager - informer/cache is started dynamically for the new cluster
- request payload includes
agent- request payload includes
agentEndpointand optionalagentToken - Gin uses the stored endpoint and token to pull summary, list resources, and send execution actions
- request payload includes
Relevant routes:
POST /api/v1/clustersPOST /api/v1/clusters/:clusterID/workloads/deployments/restartPOST /api/v1/clusters/:clusterID/workloads/deployments/scale
Cluster Resource-Monitoring Connections
Prometheus and Grafana connections belong to a cluster, not to Soha’s global runtime configuration. A cluster record may store:
- Prometheus URL and bearer token
- Prometheus cluster label
- Grafana base URL
Resource and virtualization pages must send clusterId; the backend then selects that cluster’s connection. The server currently uses a 60-minute query range and 60-second step as query defaults. This allows different clusters to use different monitoring stacks and lets the UI distinguish not configured, no data, and query failure. The global Runtime Configuration page must not display these connection fields.
The API does not return the bearer token after it is saved, but the current cluster implementation stores it in database-backed cluster connection metadata without field-level encryption. Protect database access, backups, replicas, and diagnostic exports accordingly. Do not assume security.credential_encryption_key protects this field.
Monitoring Runtime
Alert ingestion and alert governance belong to the Monitoring Workbench. Alertmanager, Grafana Alerting, and Generic Webhook integrations, tokens, routing, notifications, silences, healing, and on-call settings are maintained under /monitoring-workbench/integrations and their detail pages.
Soha keeps only the system-level switch and secret required by its monitoring ingress in startup configuration:
monitoring:
enabled: true
webhook_token: soha-123456789012345678901234567890Prometheus/Grafana resource connections do not belong here. They are cluster-scoped because each cluster may use a different Prometheus/Grafana. Resource pages select the connection by clusterId; do not place cluster Prometheus URLs, bearer tokens, default query ranges, or Grafana URLs in the global Runtime Configuration page.
Current persistence layout:
alert_events: normalized inbound alert inventorynotification_channels: downstream channel definitionsevent_stream: normalized alert activity for the unified event center
Current public or protected routes:
POST /api/v1/integrations/alerts/webhookGET /api/v1/monitoring/summaryGET /api/v1/alertsGET /api/v1/notification-channelsPOST /api/v1/notification-channelsPUT /api/v1/notification-channels/:channelID
Application Registry And GitLab
Application management now uses one PostgreSQL-backed registry table:
applicationsbuild_recordsai_sessionsai_messagesai_root_cause_runsai_data_sourcesai_analysis_profilesai_automation_policiesai_agent_runs
GitLab is a global code-source/system integration, not a delivery-workbench-only setting. It is now managed only through the Code Sources list and GitLab detail page; the server no longer reads or imports a gitlab block from config.yaml.
An integration record includes a display name, GitLab URL, encrypted token, default group, timeout, enablement, connection-test status, and audit metadata. Delivery and other workbenches reference the Provider ID and repository coordinates.
When enabled, soha serves these routes from the backend:
GET /api/v1/applicationsPOST /api/v1/applicationsGET /api/v1/applications/:applicationIDPUT /api/v1/applications/:applicationIDDELETE /api/v1/applications/:applicationIDGET /api/v1/integrations/gitlab/projectsGET /api/v1/integrations/gitlab/branchesGET /api/v1/integrations/gitlab/tags
Delivery compatibility and AI routes also exist. New console delivery actions create and confirm DeliveryPlan; the direct build trigger remains for API compatibility:
POST /api/v1/delivery/plansPOST /api/v1/delivery/plans/:planID/confirmGET /api/v1/buildsPOST /api/v1/builds/triggerGET /api/v1/copilot/insightsGET /api/v1/copilot/sessionsGET /api/v1/copilot/sessions/:sessionIDPOST /api/v1/copilot/sessionsPATCH /api/v1/copilot/sessions/:sessionIDDELETE /api/v1/copilot/sessions/:sessionIDGET /api/v1/copilot/sessions/:sessionID/messagesPOST /api/v1/copilot/sessions/:sessionID/messagesPOST /api/v1/copilot/sessions/:sessionID/analyzeGET /api/v1/copilot/agent-providersGET /api/v1/copilot/agent-runsPOST /api/v1/copilot/agent-runs/claimPOST /api/v1/copilot/agent-runs/callbackPOST /api/v1/copilot/agent-runs/tool-call
The browser never talks to GitLab directly. Tokens stay in encrypted backend-managed integration records.
YAML Compatibility And Migration Boundary
Move configuration with an import, deprecation, then deletion sequence:
- Legacy
auth.oidc.*remains a YAML/runtime compatibility path where supported. It is not imported into the settings-center database and does not create a Login Settings provider; recreate the provider there before removing the YAML values. - Legacy
gitlab.*is imported once when no GitLab connection exists. After import, use the Code Sources page as the write surface and retain YAML only as the documented startup fallback during the compatibility window. - Old global Prometheus/Grafana Helm values are no longer rendered. Before upgrading, record the non-secret connection settings through the existing secure operational process, then configure each target cluster through its cluster settings/API and enter credentials there. A global connection cannot be assigned automatically in a multi-cluster installation.
- Do not add new features that depend on legacy YAML or Helm fields. Remove remaining compatibility fields only after their documented migration path has been verified.
Do not delete fields before migration records, encryption-key handling, and multi-replica consistency are in place. Startup topology, network trust boundaries, system secrets, and infrastructure backends remain deployment-managed.
Agent Runner Config
The standalone soha-agent binary can also claim AI Agent Runtime work from the control plane. The server and agent must share the same runtime.execution_runner_token / control_plane.bearer_token value.
Minimal Hermes runner config:
control_plane:
enabled: true
base_url: http://127.0.0.1:8080
bearer_token: soha-123456789012345678901234567890
agent_id: local-agent
agent_runtime:
enabled: true
worker_id: local-hermes-runner
provider_ids:
- hermes
provider_kinds:
- hermes
hermes_command: hermes
providers:
hermes:
command: hermes
args:
- chat
- -Q
prompt_arg: -q
skill_arg: ""
provider_skill_arg: -s
workspace_root: ./.soha/agent-runtime
poll_interval: 5sThe AI workbench and automation policy choose agentProviderId; the runner only executes matching AgentRun rows. Provider-specific commands and workspaces belong in agent config, not in browser payloads. Hermes skill arguments should come from AgentSkillBinding.providerSkillRef, with soha skill ids used only as fallback. External providers should invoke soha read-only tools through /api/v1/copilot/agent-runs/tool-call, which validates the runner token, per-run callback token, and permission-filtered AgentRun.toolBindings snapshot before recording ToolExecution. The built-in Hermes/CLI POC prefetches a small read-only tool context into the provider prompt: events, logs, metrics, delivery releases, delivery builds, execution tasks, platform resource snapshots, Docker operation/service context, virtualization operations, alerts, and OnCall route resolution are currently executable. Trace routes and query DTOs remain for compatibility, but trace queries are unavailable until a future external provider integration is delivered. The generic external logs.query capability currently covers durable cluster logs only. Prefetch and provider-visible tools still depend on the run’s toolset selection, including exact adapter ids such as logs.v1 or source-kind aliases such as logs. Richer provider-native or MCP client tool protocols should still terminate at the same soha tool-call gateway.
CLI v2 authentication policy (pending release combination)
Enable new session endpoints only after verifying matching Core/Web/CLI versions. They default to disabled. Key authentication and existing Web/App login remain separate.
auth:
cli:
enabled: true
public_url: https://soha.example.com
allow_password: false
device_ttl: 5m
poll_interval: 5s
access_ttl: 15m
idle_ttl: 168h
absolute_ttl: 720h
create_limit: 10
source_pending_limit: 5
active_grant_limit: 1000
source_attempt_limit: 20
user_attempt_limit: 5public_url is the externally reachable console URL. For a path deployment, include the prefix and align the Web build, reverse proxy and provider callbacks. HTTPS is required outside loopback. Device TTL is 30 seconds to 10 minutes, polling is 1 to 30 seconds, and session TTLs must satisfy access ≤ idle ≤ absolute ≤ 365 days. Budgets are between 1 and 10000. Layered limits and state are persisted in PostgreSQL and do not reset on restart.
Every replica must share the persistent database and retain the same JWT and credential-encryption keys after restart. Helm upgrades reuse these keys from the live Secret; offline rendering does not prove live lookup. Deliberate key migration requires separate backups, compatibility verification and a previous-key retention window. Replacing values directly is insufficient. Disabling auth.cli.enabled stops new device/password sessions while retaining refresh and revocation for existing sessions.
See CLI login for usage, file-storage risks, Key import and recovery.