OpenSteward

OpenSteward is a local-first agent orchestration workspace. It lets an operator define agents, execution configurations, durable documents/artifacts, reusable tools, and engine access for Codex, Claude, Gemini, Cursor, and local collector workflows.

The current app is a React/Vite client backed by the modular api/ service, PostgreSQL persistence, and the worker-api/ Docker execution runtime.

What Is Here

• Agent configuration UI for agents, executions, artifacts, tools, and engine setups.
• Runtime session UI for starting and monitoring agent work.
• Tool setup pages with stored server-side credentials and tool manuals.
• Portable sync bundles for moving local agents, sessions, documents, artifacts, and configuration into another OpenSteward environment.
• Admin/debug APIs for production monitoring without direct database or container access.

Local Development

The supported local stack uses the modular API and worker services from the root Compose file:

cp .env.example .env

Generate two independent random values and paste them into .env as OPENSTEWARD_CREDENTIAL_KEY and OPENSTEWARD_WORKER_API_TOKEN:

node -e "console.log(require('node:crypto').randomBytes(48).toString('base64'))"

Then start the stack:

docker compose up -d --build

Then open:

• App: http://localhost:5173
• API: http://localhost:3010
• Worker API: http://localhost:3020

This starts:

• app: Vite client, proxied to api.
• api: modular API and PostgreSQL database.
• worker-api: worker API plus isolated Docker daemon for execution runs.

Useful commands:

docker compose restart api
docker compose logs -f api
docker compose down

If you run the modular API directly on the host while using the Compose database, set the database name explicitly:

$env:DATABASE_URL='postgres://opensteward:opensteward@localhost:55420/opensteward'
npm run dev:api

The full execution runner path is easiest through Compose because api talks to worker-api, which owns the isolated Docker runner.

Node Commands

npm install
npm run dev:client
npm run dev:api
npm run build
npm run test:run

Install the managed-agent CLI from this checkout:

npm link
opensteward agents
opensteward chat <agent-key>

Use opensteward run <agent-key> "..." for a one-shot, scriptable turn and opensteward resume <session-id> to continue the same managed session. See Use Managed Agents From The CLI.

npm run lint currently scans some non-app/generated areas too; prefer targeted lint while that is being cleaned up, for example:

npx eslint src/App.jsx src/lib/api.js

Important Environment Variables

• DATABASE_URL: PostgreSQL connection string. Compose sets this to the API database service.
• OPENSTEWARD_DB_POOL_MAX: connections the API keeps for ordinary queries and transactions, defaults to 20.
• OPENSTEWARD_DB_CONNECTION_TIMEOUT_MS: maximum wait for a free pooled connection, defaults to 10000; 0 waits forever, which is what used to turn an exhausted pool into a silent stall.
• OPENSTEWARD_DB_IDLE_TIMEOUT_MS: how long an unused pooled connection is kept, defaults to 30000.
• OPENSTEWARD_DB_STATEMENT_TIMEOUT_MS: server-side statement_timeout for pooled connections, defaults to 60000; 0 leaves it to the server. Migrations opt out on their own.
• OPENSTEWARD_DB_LOCK_POOL_MAX: connections reserved for session advisory locks, defaults to 30. Locks are held for as long as the work they guard - an execution turn lock spans the whole worker call - so they never share capacity with queries. Size it for concurrent agent turns per replica times the lock nesting depth of a turn, and use OPENSTEWARD_DB_LOCK_CONNECTION_TIMEOUT_MS, OPENSTEWARD_DB_LOCK_IDLE_TIMEOUT_MS and OPENSTEWARD_DB_LOCK_STATEMENT_TIMEOUT_MS for its timeouts.
• OPENSTEWARD_CREDENTIAL_LOCK_HOLDER_LIMIT: advisory lock chains one replica admits at once, defaults to a third of the lock pool so every admitted chain can still take its nested locks.
• PORT: API port, defaults to 3000.
• VITE_API_PROXY_TARGET: Vite dev proxy target, defaults to http://127.0.0.1:3010; the Docker app service points at the Compose api service.
• OPENSTEWARD_API_RUNTIME_DIR: modular API runtime workspace directory, defaults to runtime.
• OPENSTEWARD_WORKER_API_URL: worker API URL used by api to dispatch executions.
• OPENSTEWARD_CORS_ORIGINS: optional comma-separated exact browser origins for split-origin deployments. Same-origin requests need no CORS entry; wildcard access is never the default.
• OPENSTEWARD_WORKER_API_ALLOWED_URLS: optional comma-separated list of additional exact Worker API base URLs. An origin-wide wildcard is not accepted.
• OPENSTEWARD_WORKER_API_TOKEN: random deployment-owned bearer token shared only by the API and Worker API. It must contain at least 32 bytes.
• OPENSTEWARD_WORKER_DISPATCH_CLIENT_TIMEOUT_MS: maximum API wait for the Worker API to prepare and claim a worker session, defaults to 1800000 (30 minutes).
• OPENSTEWARD_WORKER_DISPATCH_STALE_BUFFER_MS: safety buffer added before an unclaimed worker dispatch is eligible for orphan reconciliation, defaults to 300000 (5 minutes).
• OPENSTEWARD_ORPHANED_WORKER_EXECUTION_STALE_SECONDS: requested orphan threshold, defaults to 300; positive values are clamped to at least the worker dispatch timeout plus its stale buffer.
• OPENSTEWARD_AGENT_TURN_TIMEOUT_MS: maximum model turn duration used by both the API and Worker API, defaults to 3600000 (60 minutes). Configure the same value for both services.
• OPENSTEWARD_CREDENTIAL_LOCK_WAIT_TIMEOUT_MS: maximum API wait for another replica to release an agent-engine credential lock, defaults to 2100000 (35 minutes).
• OPENSTEWARD_WORKER_TRACE_SYNC_CLIENT_TIMEOUT_MS: maximum API wait for post-turn worker trace and artifact finalization, defaults to 1800000 (30 minutes).
• OPENSTEWARD_WORKER_RUN_WORKSPACE_DEADLINE_MS: aggregate worker deadline for persisting one run workspace, defaults to 120000 (2 minutes).
• OPENSTEWARD_WORKER_RUN_WORKSPACE_MAX_BYTES: maximum aggregate bytes persisted from one run workspace, defaults to 67108864 (64 MiB).
• OPENSTEWARD_WORKER_RUN_WORKSPACE_MAX_ENTRIES: maximum aggregate files and directories persisted recursively from one run workspace, defaults to 256.
• OPENSTEWARD_WORKER_RUN_TRANSCRIPT_MAX_BYTES: maximum bytes kept per engine transcript file under /workspace/run/agent-sessions, defaults to 33554432 (32 MiB). Transcripts are truncated to this size and never count toward the run workspace limits, so a run that read many pages or images still delivers its artifacts.
• Agents that opt into session environments can declare session_environment.restore ({ "command": "node /workspace/run/memory/start-preview.mjs", "timeout_seconds": 600 }). When someone starts the environment of a session that previously published a runtime preview, the platform runs that command inside the new runner container (as /bin/sh -lc, with the container's environment) so the preview comes back without a chat turn. The command is responsible for re-registering its ports through the tool-action recorder; progress and failures show in the session's environment bar. Sessions that never had a preview skip it, and POST .../runner/start accepts restore_preview: false to skip it once.
• Inside a session environment the container receives OPENSTEWARD_API_URL and OPENSTEWARD_API_KEY: a personal API key of the person who started the session, valid for two days and revoked when the environment stops. Starter scripts and agent tooling use it as a bearer token to call the platform, which sign-in mode requires for every route except tool-action reporting.
• OPENSTEWARD_SESSION_ENVIRONMENT_IDLE_TIMEOUT_MS: managed session environment inactivity limit, defaults to 3600000 (1 hour). Agent work and real session UI interaction renew the lease; passive status polling does not.
• OPENSTEWARD_SESSION_ENVIRONMENT_REAPER_INTERVAL_MS: how often the API checks for expired managed environments, defaults to 60000 (1 minute).
• OPENSTEWARD_WORKER_EXECUTION_WORKSPACE_RETENTION_MS: retention for inactive worker execution workspaces, defaults to 21600000 (6 hours).
• OPENSTEWARD_WORKER_EXECUTION_WORKSPACE_MAX_BYTES: total worker execution workspace budget, defaults to 2147483648 (2 GiB). Oldest inactive work is removed first; active work is never removed to satisfy the budget.
• OPENSTEWARD_WORKER_BUILDKIT_KEEP_STORAGE: Docker BuildKit cache budget, defaults to 2GB. Shared content-addressed layers remain reusable across sessions within this bound.
• OPENSTEWARD_RUNTIME_PREVIEW_ENABLED=1: explicitly enables the experimental runtime proxy. It is off by default. When enabled, the worker automatically registers only artifact-requested ports that Docker confirms are published by a running container owned by the same execution and session.
• OPENSTEWARD_RUNTIME_PREVIEW_MAX_RESPONSE_BYTES: maximum buffered upstream response accepted by the runtime proxy, defaults to 33554432 (32 MiB) and is capped at 100 MiB.
• OPENSTEWARD_CREDENTIAL_KEY: deployment-owned key used to encrypt stored credentials. Production rejects missing, short, and published development keys.
• OPENSTEWARD_ADMIN_API_KEY: protects admin/operator routes.
• OPENSTEWARD_SYNC_API_KEY: protects sync import/export routes.
• OPENSTEWARD_SYNC_ALLOW_TOOL_SECRETS=1: allows sync export of tool secrets only when the request also opts in.
• OPENSTEWARD_PUBLIC_APP_URL: exact public application origin used by OAuth and setup flows.
• OPENSTEWARD_PUBLIC_URL: optional public origin (no trailing slash) used to build artifact share links. When unset, links derive from X-Forwarded-Proto/X-Forwarded-Host or the request Host.
• OPENSTEWARD_AUTH_MODE: none (default, every request acts as the local operator user) or oidc (browser sign-in through an OpenID Connect provider). For direct Google login, set OPENSTEWARD_OIDC_ISSUER=https://accounts.google.com and the Google Web client's OPENSTEWARD_OIDC_CLIENT_ID and OPENSTEWARD_OIDC_CLIENT_SECRET. Optionally restrict company accounts per instance with OPENSTEWARD_OIDC_ALLOWED_EMAIL_DOMAINS (for example twinai.nl); Google login then checks both verified email and the signed hosted-domain claim. Empty disables these domain restrictions. Choose initial admins with verified OPENSTEWARD_ADMIN_EMAILS or stable OPENSTEWARD_ADMIN_SUBJECTS; OPENSTEWARD_ADMIN_API_KEY provides operator recovery. New users default to Member, including the first sign-in. See direct Google Workspace sign-in and user management, including Entra setup and issuer migration.

Keep an existing OPENSTEWARD_CREDENTIAL_KEY for as long as any credential rows were encrypted with it. Replacing or losing the key makes those AES-GCM records undecryptable; the current release does not provide automatic key rotation. Restore the old key or deliberately re-enter credentials before switching keys. The Worker API token is independent and can be rotated by updating the API and Worker API together, then restarting both services.

Managed deployment trust model

By default the managed API is a trusted single-operator control plane: every request acts as the local operator user and there is no RBAC. Keep it on loopback or a private network behind operator authentication; do not expose ports 3010 or 3020 directly to untrusted clients. CORS limits which browsers can read responses, but it is not authentication. With OPENSTEWARD_AUTH_MODE=oidc the API requires a signed-in user (see docs/user-authentication.md); agents and sessions are still shared by every user of the instance.

Generated HTML/XHTML/SVG previews are intentionally static: scripts, form submission, and outbound connections are blocked. Runtime application preview is disabled by default. Restoring interactive previews safely requires a dedicated cookie-less preview origin and worker/container-bound grants.

Voice and phone agents

Agents can opt into browser conversations through Codex subscription or OpenAI API, plus operator-initiated outbound SIP calls. Configure Voice and phone conversations in the agent editor, then use the session voice panel. Codex uses an authorised OpenSteward ChatGPT connection without API fallback. The API route needs a server-side OpenAI API key; phone calls additionally need a configurable SIP trunk, caller number and OpenAI outbound access. Voice is disabled by default. See Create and test a voice/phone agent.

Caller speech is retained as voice conversation data and cannot approve plans or execute business tools. An operator can prepare a separate draft follow-up session for review. Both audio tracks, transcripts and call outcomes stay under session access controls and can travel with bounded session exports. Codex uses browser WebM/Ogg capture; API calls use reflected WAV. Inbound-number routing is not included.

Sharing artifacts

A session artifact can be shared from Share in the artifact viewer. When enterprise OIDC login is enabled (Google, Zitadel, or another provider), Colleagues is selected by default; the option is labelled with the allowed mail domains (for example Colleagues (@twinai.nl)) or, without a domain restriction, with the sign-in provider. Recipients must authenticate with an identity accepted by this instance's provider and domain rules. Choose Public to allow anyone with the link to view and download without signing in; the dropdown shows a small warning before creating or saving public access. For existing links, choose an option and click Save access; the URL stays the same. Revoke the link from the popover to stop access; creating it again issues a fresh token.

Existing links without a saved permission inherit the instance default, including company access when enterprise login is enabled. New company links store their restriction and stay locked if sign-in is later disabled. Explicit public choices remain public. Instances without enterprise login retain public sharing as their default. Every link shows the latest artifact content. A company share grants access to that artifact, not to its parent session or agent.

• Share path: /s/<token> renders markdown/text in a script-free page, serves HTML in the static preview sandbox, and redirects binaries to /s/<token>/raw (add ?download=1 for an attachment). Pages, raw bytes, and downloads all enforce the selected access. The same routes are mounted at /api/v1/shares/<token> and /api/shares/<token>.
• Deployments behind an auth proxy (for example Traefik basic auth) must exempt /s/ and forward it to the API so public links can work. The API itself enforces company-link authentication. Anonymous company viewers receive a relative redirect to /api/v1/auth/login and return to the share or download after normal OIDC sign-in; the share route adds no public-origin configuration requirement. Invalid explicit personal API keys receive 401 without a login redirect.
• Set OPENSTEWARD_PUBLIC_URL to the public origin when the API cannot see the external host through forwarded headers.

Sync API

OpenSteward can export portable JSON sync bundles for moving local agent/session work into another OpenSteward environment.

• GET /api/sync/agents/:id/export exports an agent, its configuration, documents, and sessions.
• GET /api/sync/sessions/:id/export exports the owning agent configuration plus one session.
• POST /api/sync/import imports a bundle into the target environment, preserving UUIDs so repeated syncs update the same records.

Useful query parameters:

• include_sessions=false exports only agent configuration.
• session_ids=<uuid>,<uuid> limits an agent export to selected sessions.
• include_documents=false omits document file payloads.
• include_artifact_files=false omits artifact file payloads.
• dry_run=1 on import validates and summarizes the bundle without writing.

Set OPENSTEWARD_SYNC_API_KEY in the target environment to require an app-level key for all sync import/export routes. Clients can send it as either:

Authorization: Bearer <key>

or:

X-OpenSteward-Sync-Key: <key>

Tool credentials are redacted by default. To export stored tool secrets, both the request query include_tool_secrets=true and server environment variable OPENSTEWARD_SYNC_ALLOW_TOOL_SECRETS=1 must be set. Active local runtime state is made inert on import: running sessions become waiting_for_user, active execution runs become cancelled, and active generator sessions become stopped.

Tool Library API

OpenSteward can reuse agent-owned tools without sharing credentials between agents.

• GET /api/tool-library/tools searches tools across agents and returns source agent, linked execution context, run count, last-used time, source import type, and non-secret tool configuration.
• POST /api/tool-library/tools/:id/copy copies one tool into a target agent.

Useful query parameters for search:

• query=<text> matches tool name, source agent, type, env vars, description, and instructions.
• tool_kind=<kind> filters by tool type.
• agent_id=<uuid> limits results to one source agent.
• include_archived=1 includes archived tools.
• limit=<number> caps results, up to 120.

Copy request body:

{
  "target_agent_id": "<agent uuid>",
  "mode": "copy_configuration"
}

Use mode: "copy_configuration" to reuse install/setup/verify commands, instructions, env var names, JSON config, and source import rules. Use mode: "fresh_configuration" to create the same tool kind from defaults with new configuration. Stored token values, OAuth secrets, cookies, and browser state are never copied.

Admin API

Set OPENSTEWARD_ADMIN_API_KEY to protect operator/debug routes. If it is not set, OpenSteward falls back to OPENSTEWARD_SYNC_API_KEY; if neither key is set, admin routes are open for local development.

Clients can authenticate with:

Authorization: Bearer <key>

or:

X-OpenSteward-Admin-Key: <key>

The admin API is intentionally structured JSON rather than raw SQL:

• GET /api/admin/status returns app version, build/runtime settings, limits, and key configuration booleans.
• GET /api/admin/diagnostics returns DB/storage health, queue counts, engine setup state, recent errors, and recent admin audit events.
• POST /api/admin/diagnostics/run runs diagnostics and can refresh engine metadata.
• GET /api/admin/sessions/:id/debug returns the full session debug snapshot: config, messages, events, runs, artifacts, artifact versions, documents, evaluations, and checklist items.
• GET /api/admin/execution-runs lists runs with optional session_id, status, and limit filters.
• GET /api/admin/execution-runs/:id/debug returns one run with related session/config/events/artifacts/evaluations and optional include_logs=1.
• GET /api/admin/execution-runs/:id/logs returns runner logs.
• POST /api/admin/execution-runs/:id/cancel marks a non-terminal run as cancelled.
• POST /api/admin/execution-runs/:id/retry queues a retry using the same execution configuration.
• GET /api/admin/agents/:id/config returns the full agent configuration.
• PATCH /api/admin/agent-tools/:id updates a tool using the same payload shape as /api/agent-tools/:id.
• POST /api/admin/sessions/:id/context-document/refresh rebuilds session context documents.
• GET /api/admin/session-artifacts/:id/history and /download inspect artifacts.
• GET /api/admin/documents/:id/download downloads stored documents.

Admin mutations write to admin_audit_events.

Documentation Map

• docs/README.md: documentation entry point split into tutorials, how-to guides, reference, and understanding.
• docs/tutorials/: guided first-time paths for building, evaluating, repo-managing, and model-comparing agents.
• docs/how-to/: task-oriented guides for local runs, tools, executions, imports, debugging, and model changes.
• docs/reference/: technical contracts for source tree, API, data model, sessions, executions, artifacts, tools, definitions, and evaluations.
• docs/understanding/: architecture explanations for lifecycle, repo-managed agents, evaluations, and security.
• docs/new-agent-development.md: older single-page guide for building and maintaining agents.
• docs/agent-configuration-plan.md: current agent/execution/artifact configuration model.
• docs/agent-runtime-runner-plan.md: runtime and Docker runner design.
• docs/codex-engine.md, docs/claude-engine.md: engine setup notes.
• docs/gmail-oauth-tools.md: Gmail OAuth tool setup.
• docs/tool-manual-artifacts.md: durable tool manuals.
• docs/tekton-fluxcd.md: production build/deploy flow.