Platform

SSE Streaming Flow

Processor Architecture

The chat-proc service is the execution side of the platform. After ingress admits and enqueues a request, the processor claims it, loads the target bundle, executes the workflow, and streams results back through the relay communicator.

Bundle State: What Lives Where

This split is intentional: only deployment-scoped bundle state belongs to descriptor authority and bundle export. User-scoped state stays outside bundles.yaml and bundles.secrets.yaml.

When isolated execution runs in Docker or Fargate, chat-proc ships the descriptor authority to the supervisor as KDCUBE_RUNTIME_*_YAML_B64 payloads. The supervisor materializes those descriptors and uses the same settings/secrets APIs as proc; generated code receives only the filtered executor environment. Bundle runtimes can set descriptor_payload_scope: active_bundle to filter bundles.yaml and bundles.secrets.yaml to the active caller bundle before packaging.

Task Queue Model

The processor uses Redis Lists as its task queue. Ingress pushes task payloads with LPUSH; processor workers claim them with BRPOPLPUSH, atomically moving the item from a ready queue to an inflight queue. This gives FIFO ordering within each lane.

Tasks are partitioned into user-type lanes — privileged, paid, registered, and anonymous. Workers rotate fairly across lanes so no single tier starves the others. Each claimed task is protected by a per-task Redis lock (SET NX EX) to prevent duplicate processing.

Task Orchestrator And Background Job Stream

Interactive chat turns are only one kind of work. The processor also runs a sibling background job stream for ready work produced by cron scans, widget operations, admin actions, or bundle-specific schedulers. These jobs use Redis Streams, not the chat Redis List queue.

The stream does not decide when work is due. A producer first creates any durable bundle-owned domain record, then enqueues a ready-work envelope. The processor fairly polls chat work and background work, claims stream messages through a consumer group, builds a normal bundle runtime context, and invokes the bundle's async @on_job handler.

The platform treats metadata and payload as routing/runtime context plus bundle-owned data. It does not understand bundle-specific job semantics. That boundary lets bundles implement scheduled reports, task executions, mailbox processing, or other domain work without adding a new platform service for each job type. See the source design note: jobs-stream-README.md.

Bundle Loader & Lifecycle

Processor workers load bundles through a registry + singleton cache model:

1. On startup (and on bundle-update broadcasts), the worker rebuilds its in-memory bundle registry.
2. At request time, the registry resolves the bundle to a concrete path. Module/singleton cache keys are based on the resolved path.
3. Built-in example bundles are merged into the registry and copied to shared storage with a versioned path (/bundles/{bundle_id}__{ref}__{sha}).
4. On update, loader caches are cleared. New requests use the new path; already-running turns continue on the previously loaded path.

This means bundle updates are zero-downtime — running work is never affected, and new work picks up the latest version automatically.

Execution Pipeline

Once a task is claimed, execution follows a fixed sequence:

1. Receive — claim the task from the ready queue via BRPOPLPUSH and acquire the per-task lock.
2. Validate — materialize ChatTaskPayload, build ServiceCtx and ConversationCtx.
3. Load bundle — resolve the target bundle through the registry, load/reuse the singleton.
4. Execute — run the bundle handler under task timeout, accounting binding, lock renewal, and started-marker renewal. On ECS, scale-in protection is enabled for the duration.
5. Stream results — the bundle emits events through the ChatCommunicator; the relay forwards them via Redis pub/sub to ingress, which delivers them to the client over SSE.

On success the inflight claim is acked, conversation state moves to idle, and conv_status is emitted. On failure the conversation is set to error and the client receives a chat.error event.

State Management & Recovery

The processor distinguishes two recovery cases based on the started marker:

• Pre-start claims (lock expired, no started marker) — safe to requeue. The inflight reaper moves the item back to the ready lane.
• Started tasks (lock expired, started marker present) — not replayed. The conversation is set to error and the client is notified with turn_interrupted. This prevents duplicate side effects from partial execution.

The started marker intentionally outlives the claim lock so that a hard worker restart cannot accidentally let both leases expire and trigger an unsafe replay.

Continuation Mailbox

When a user sends a message to a conversation that is already executing, ingress stores it in a per-conversation ordered mailbox in Redis rather than the main ready queue. The active workflow can inspect this mailbox via the ConversationContinuationSource API (peek_next_continuation(), take_next_continuation()). If the bundle does not consume the item, the processor promotes exactly one pending item back into the ready queue after the current turn completes.

Operations API Surface

The processor exposes REST endpoints that do not require SSE:

• /integrations/bundles/{tenant}/{project}/{bundle_id}/operations/{op} — bundle-defined operations called directly by UI widgets.
• Admin endpoints for bundle registration, property management, secrets injection, and cleanup.

These endpoints run inside the processor because they need access to the loaded bundle singleton and its runtime context. This is what makes bundle-owned UI surfaces practical: widget UI or main UI can call directly into the live bundle rather than going through a second app server. Frontends may embed those surfaces, but embedding is a client display choice; KDCube serves the bundle UI and its APIs.

Runtime Availability Enforcement

The processor treats enabled_config as live runtime policy, not build-time metadata. On every inbound call it resolves the bundle-level and resource-level dot paths against the effective bundle props before dispatching the bundle method.

• Bundle disabled: operations, widgets, and MCP endpoints return 404; scheduled jobs for that bundle are not reconciled.
• Resource disabled: only that operation, widget, MCP endpoint, or cron job is suppressed.
• No switch or missing prop: the surface stays enabled.

HTTP enforcement lives in the processor integration layer; cron enforcement lives in the bundle scheduler. This is why a bundle-props update can change surface availability immediately for new requests without rebuilding or redeploying the bundle.

Communicator Integration

The processor uses the relay communicator pattern: bundle events are published to Redis pub/sub channels, ingress subscribes and fans them out over SSE to connected clients. This decouples execution from delivery — the processor never holds SSE connections directly, and horizontal scaling of proc replicas does not affect client connectivity.

Recorded Events and Isolated Tool Handoff

The communicator can also record selected post-firewall envelopes into scoped, bounded buffers. Recording is runtime-local while execution is active, but portable recording scopes are included in COMM_SPEC when the processor launches a platform child runtime.

The normal isolated-tool pattern is: child runtime records, child writes comm_recorded_events.json, host merges the side file into the host communicator, and the host sends the merged batch through the configured sink. This covers TOOL_RUNTIME[tool_id] = "local" as well as Docker/Fargate tool execution when the runtime output directory is returned. Sink callbacks remain host-side unless the child explicitly configures its own sink.

Process Topology

Each Uvicorn worker in chat-proc is an independent queue consumer with its own Redis pool, Postgres pool, bundle registry, inflight reaper, and cleanup loop. All workers across all replicas compete for the same Redis task queues. There is no sticky worker-to-conversation affinity — any worker can claim any task, and conversation ownership is per-turn, not per-instance.