Architecture

This page explains how ByteChef executes workflows under the hood. It is written for contributors and component authors who want to understand the moving parts: the Atlas engine that runs jobs, the trigger and webhook machinery that starts them, the message broker that connects everything, and the Component SDK whose definitions become both editor UI and executable tasks.

If you only want to build a component, start with the Developer Guide and the Component Specification; come back here when you want to know what happens after you hit Run.

The big picture

ByteChef is a modular Spring Boot application. Every box below is a set of Gradle modules under server/libs/; the arrows are either direct calls or messages on the broker.

Three properties of this design are worth internalizing early:

1. Configuration and execution never mix. A Workflow definition is immutable design-time data; a Job is runtime state referencing it. They live in different modules with different repositories.
2. Everything between coordinator and worker is a message. Atlas components communicate exclusively through named broker routes, so the same code runs as one JVM (in-memory broker) or as separately scaled services (Redis/Kafka/RabbitMQ) with zero code changes.
3. Components are data first, code second. A component definition is a declarative object the platform maps into editor UI, REST models, and task handlers. The perform function is just one (optional!) attribute of it.

Module layout

The server modules form strict layers — each layer only depends downward:

Most modules follow an -api / -service (or -impl) / -config split: interfaces and domain objects in -api, implementations in -service, Spring wiring in -config. Depending on the -api module alone is what lets a consumer be wired against a different implementation without touching its own code.

Configuration: workflows as data

The design-time side lives in atlas-configuration. A Workflow is parsed from JSON (or YAML) by WorkflowReader into WorkflowTask objects. Where definitions are stored is pluggable via WorkflowRepository implementations: JDBC (the default for user workflows), classpath, filesystem, and Git.

{
    "label": "Daily Invoice Capture",
    "inputs": [
        { "name": "hourToRun", "type": "integer", "required": true }
    ],
    "triggers": [
        {
            "name": "trigger_1",
            "type": "schedule/v1/everyDay",
            "parameters": { "hour": "${hourToRun}", "minute": 0 }
        }
    ],
    "tasks": [
        {
            "name": "searchEmail_1",
            "type": "googleMail/v1/searchEmail",
            "parameters": { "q": "invoice has:attachment" }
        }
    ]
}

Two things in this JSON drive everything else:

• The task type string — component/vVERSION/operation. It is parsed by WorkflowNodeType.ofType("googleMail/v1/searchEmail") into (name=googleMail, version=1, operation=searchEmail) and doubles as the task-handler registry key at execution time.
• ${...} expressions — evaluated lazily against the accumulated job context, not at save time (see Expression evaluation).

Above Atlas, the automation layer (automation-configuration) adds the product concepts: projects version workflows, deployments bind a published project version to an environment, per-workflow connections and inputs. Enabling a deployment workflow is what activates its triggers.

Execution: the Atlas engine

The runtime side lives in atlas-execution (state), atlas-coordinator (orchestration), and atlas-worker (task execution).

Core domain

Task outputs, job outputs, and context values are not stored inline in PostgreSQL rows. They go through TaskFileStorage, which serializes them to the file-storage subsystem and stores only a FileEntry handle on the entity. This keeps rows small no matter how large a task's payload is.

How a job runs

Every hop below is an event on a named broker route — the class names and route names are real, so you can grep for them.

Step by step:

1. Launch. A caller (deployment trigger, webhook, editor test run, API) invokes JobFacade.createJob(JobParametersDTO). The job row is created, the initial context (the job inputs) is pushed, and a StartJobEvent is published.
2. Event-to-message bridge. Atlas publishes plain Spring application events; MessageEventListener (a @TransactionalEventListener(AFTER_COMMIT)) forwards each one onto its broker route. Publishing after commit guarantees a consumer never sees a message for state that isn't in the database yet.
3. Coordinator starts the job. TaskCoordinator.onStartJobEvent marks the job STARTED and calls JobExecutor.execute(job), which reads the current context, builds the next TaskExecution from workflow.getTasks().get(job.getCurrentTask()), evaluates its ${...} parameters, persists it, and hands it to the dispatcher.
4. Dispatch. The primary dispatcher is a TaskDispatcherChain. Control-flow dispatchers (condition, loop, ...) get first refusal; if none claims the task, DefaultTaskDispatcher publishes it to the worker route. Tasks can target named worker pools: a task with "node": "gpu" routes to task.gpu_task_execution_events instead of task.default_task_execution_events.
5. Execute. TaskWorker resolves a TaskHandler through the TaskHandlerRegistry — keyed by the exact task type string (googleMail/v1/searchEmail) — invokes it (24h default timeout), stores the output via TaskFileStorage, and publishes TaskExecutionCompleteEvent.
6. Complete and advance. Back on the coordinator, the TaskCompletionHandlerChain gives control-flow completion handlers first refusal; DefaultTaskCompletionHandler handles top-level tasks: it merges the output into the job context under the task's name, then either dispatches the next task or completes the job (evaluating declared workflow outputs against the final context).

Stop and resume follow the same pattern (StopJobEvent on task.stop_job_events, ResumeJobEvent on task.resume_job_events), as do errors (task.error_events).

Expression evaluation

${...} references are resolved by SpelEvaluator (Spring SpEL with a custom map property accessor, in core/evaluator) — plus a formula syntax where a parameter value starting with = is evaluated as an expression with built-in functions (concat, join, range, now, uuid, ...).

Evaluation happens at dispatch time, on the coordinator, against the context accumulated so far — that's why ${searchEmail_1.id} works in task 3 but not in task 1. Control-flow dispatchers register deferred evaluation keys (e.g. condition defers caseTrue/caseFalse) so expressions inside not-yet-taken branches survive verbatim until the branch is actually dispatched.

Control flow: task dispatchers

Control flow is not built into the engine — each construct is a pluggable extension in server/libs/modules/task-dispatchers/ following one pattern:

Each dispatcher module contributes three pieces, registered through factory beans:

• a TaskDispatcher that recognizes its task type, evaluates its parameters (which branch? how many iterations?), and dispatches child TaskExecutions back through the chain;
• a TaskCompletionHandler that intercepts child completions and decides what's next (next iteration, next branch task, or "parent done");
• a Spring configuration registering both, plus deferred-evaluation keys for its nested-task parameters.

Fan-out constructs (parallel, fork-join, each, map) additionally use the Counter service to know when all children have landed. subflow is the outlier: it launches a whole child Job linked via parentTaskExecutionId.

The message broker

The broker abstraction lives in core/message: a MessageRoute (name + exchange), a MessageBroker.send(route, message), and per-provider implementations selected by bytechef.message-broker.provider — memory (default), amqp (RabbitMQ), jms, kafka, redis, and AWS SQS (EE).

Because coordinator and worker only ever talk through routes, scaling out means moving off the in-process broker to a shared one and running more instances — the Atlas code is identical either way. Workers subscribe to routes by node name (bytechef.worker.task.subscriptions), which is how a task can be pinned to a dedicated queue.

There is one deliberate bypass: JobSyncExecutor (platform/platform-job-sync) hand-wires a private coordinator + worker around an in-process broker to run a workflow synchronously start-to-finish. That's what powers the editor's test runs and streamed executions, without touching the shared queues.

Triggers

Triggers start jobs. A component trigger declares one of these types (TriggerDefinition.TriggerType in the SDK):

The Schedule component is a LISTENER: its listenerEnable registers a cron job with the platform scheduler, and each cron firing emits a trigger event.

Lifecycle

When a deployment workflow is enabled, ProjectDeploymentFacadeImpl.enableWorkflowTriggers builds a WorkflowExecutionId for each trigger and calls TriggerLifecycleFacade.executeTriggerEnable, which branches on the type:

The WorkflowExecutionId deserves a note: it is the base64-encoded tuple tenantId : type : jobPrincipalId : workflowUuid : triggerName. It is the webhook URL path segment, the scheduler job identity, and the trigger-state key — one durable address for "this trigger of this workflow in this deployment".

TriggerStateService persists whatever a trigger needs to remember between invocations: the webhook registration output for dynamic webhooks, and the closureParameters cursor for polling triggers.

The scheduler (platform-scheduler) is Quartz-backed by default (DB-persistent), with an AWS EventBridge implementation in EE. It runs three kinds of jobs: PollingTriggerJob (fires every poll period, default 5 minutes), ScheduleTriggerJob (cron, for the Schedule component), and DynamicWebhookTriggerRefreshJob (one-shot at the registration's expiration).

From trigger event to job

All trigger sources converge on the TriggerCoordinator, which mirrors the task pipeline: dispatch to a TriggerWorker over the broker, execute the trigger function, then hand the completed TriggerExecution to TriggerCompletionHandler, which creates the job(s):

• trigger output is a list and the trigger is not batch → one job per element;
• batch → one job with the whole list as the trigger's output;
• single value → one job.

The trigger's output lands in the job inputs under the trigger's name — which is why workflow expressions reference ${trigger_1.someField}.

For polling triggers the worker loops the component's poll function: each call returns PollOutput(records, closureParameters, pollImmediately); the records accumulate (safety caps: 10,000 records / 100 iterations per run) and the final closureParameters — typically a "last seen" timestamp or cursor — are persisted as trigger state and passed back on the next poll. Deduplication is cursor-based, not record-hash based.

Webhooks

The webhook endpoint is the platform's front door for external events:

Details that matter:

• Request capture. The raw HTTP request (method, headers, query, body — including multipart, stored to temp file storage) is normalized into a WebhookRequest and attached to the TriggerExecution, so the component's webhookRequest function sees the full request regardless of broker hops.
• Sync execution is opt-in per trigger (workflowSyncExecution flag on the trigger definition). The response body comes from whichever task in the workflow produced an output tagged WEBHOOK_RESPONSE — that's what the Webhook component's "response" actions set.
• JobCompletionAwaiter is how a synchronous HTTP response works when the job does not run on the receiving thread: the node that received the request parks a CompletableFuture keyed by job id and completes it when the job-status event arrives over the broker — so the job may well have executed elsewhere. A race guard checks whether the job is already terminal before parking.
• Validation flags (workflowSyncValidation, workflowSyncOnEnableValidation) let a trigger validate a request (e.g. a provider's URL-verification handshake) synchronously before any job is created.
• There is also an SSE variant (/webhooks/:id/sse) that streams job progress events back to the caller.

The Component SDK

Components are authored against sdks/backend/java/component-api — a dependency-free definition model plus the ComponentDsl fluent builder:

@AutoService(ComponentHandler.class)
public final class SlackComponentHandler implements ComponentHandler {

    private static final ComponentDefinition COMPONENT_DEFINITION = component("slack")
        .title("Slack")
        .icon("path:assets/slack.svg")
        .categories(ComponentCategory.COMMUNICATION)
        .connection(SlackConnection.CONNECTION_DEFINITION)
        .actions(SlackSendChannelMessageAction.ACTION_DEFINITION)
        .triggers(SlackAnyEventTrigger.TRIGGER_DEFINITION)
        .version(1);

    @Override
    public ComponentDefinition getDefinition() {
        return COMPONENT_DEFINITION;
    }
}

@AutoService generates the META-INF/services entry, and the platform discovers handlers with ServiceLoader. No Spring, no annotations beyond that one — a component is a plain object graph describing itself.

What Optional means in the definition interfaces

Nearly every getter on the SDK definition interfaces returns an Optional:

public interface ActionDefinition {

    String getName();                                       // mandatory - set by action(name)

    Optional<String> getTitle();                            // author may set it
    Optional<String> getDescription();
    Optional<List<? extends Property>> getProperties();
    Optional<OutputDefinition> getOutputDefinition();
    Optional<? extends BasePerformFunction> getPerform();
}

The rule: an Optional getter marks an attribute the component author may or may not have set in the fluent definition. A present value means the author called that builder method; an empty Optional means they didn't, and the platform substitutes a fallback or treats the capability as absent. (Mechanically, every DSL builder getter is literally Optional.ofNullable(field) — the field stays null until the corresponding builder method runs.)

The non-Optional getters are the mandatory attributes the DSL factory itself guarantees: component(name) sets the name, action(name) sets the action name, versions default to 1.

What "empty" concretely means, per attribute:

OutputDefinition itself illustrates the pattern one level deeper: it holds either a static schema/sample (getOutputResponse()) or a function that computes the schema at runtime against live parameters (getOutput()) — both Optional, and which one is present changes how the editor resolves the node's output shape.

From definition to editor and runtime

The SDK definition is deliberately not what the rest of the platform consumes. The platform-component module maps it into two other shapes:

• The registry (ComponentDefinitionRegistry) is the single catalog. It is fully lazy: at startup nothing is loaded; a build-time index (META-INF/bytechef/component-index.json, generated by the generateComponentIndex Gradle task) serves the components-list view from lightweight stubs, and the first deep read triggers a one-time ServiceLoader sweep. If the index is absent, the registry transparently falls back to full loading.
• The domain model (com.bytechef.platform.component.domain.*) is the flattened, Optional-free, function-free mirror used by services and serialized to the REST API — this is where the fallbacks from the table above are applied, once, eagerly.
• Task handlers: ComponentTaskHandlerProvider registers one ComponentTaskHandler per action under the key component/vN/actionName — exactly the workflow task type string. That is the whole trick connecting workflow JSON to component code.

At execution time the path is:

The perform function receives three arguments:

• Parameters inputParameters — the task's evaluated parameters (expressions already resolved by the coordinator);
• Parameters connectionParameters — the decrypted parameters of the resolved connection (token, base URI, ...);
• ActionContext context — the capability surface: context.http(...) (the HTTP client — how components call external APIs), context.json(...), context.file(...) (file storage in/out via FileEntry), context.log(...), context.data(...) (key/value storage scoped to execution / workflow / deployment / account), plus event progress publishing and suspend for human-in-the-loop pauses.

Note what this means architecturally: connections are bound to deployments and resolved server-side at dispatch — connection ids never appear in the workflow definition, and credentials never leave the facade layer except as the already-scoped connectionParameters.

OpenAPI-generated components

Components scaffolded from an OpenAPI spec (via the CLI: component init --name=... --open-api-path=...) implement OpenApiComponentHandler. Their generated actions declare only properties and metadata (HTTP method, path, where each property goes: PATH / QUERY / HEADER / BODY) — and no perform. At load time, OpenApiComponentHandlerLoader wraps each such action and injects a generic perform that assembles the HTTP call from that metadata. This is the clearest demonstration of the Optional contract: getPerform() being empty isn't an error, it's a signal the loader acts on. Hand-written modify* hooks let authors adjust the generated definitions without touching generated code.

AI & Agents

AI is built on the same component and connection model described above — an agent is a component; a deployed agent is a generated Workflow. Nothing here bypasses the Atlas engine or Component SDK.

Putting it together

One last trace, end to end — a scheduled Gmail-to-Drive workflow like the one from the configuration example:

1. Deployment enabled → TriggerLifecycleFacade sees the Schedule trigger is a LISTENER → its listenerEnable registers a Quartz cron keyed by the WorkflowExecutionId.
2. Cron fires → TriggerListenerEvent → TriggerCoordinator stores the trigger output and hands it to TriggerCompletionHandler → createJob with inputs { trigger_1: {...}, ...deployment inputs }.
3. StartJobEvent → TaskCoordinator → JobExecutor evaluates searchEmail_1's parameters against the context → DefaultTaskDispatcher → worker route.
4. TaskWorker resolves the handler under googleMail/v1/searchEmail → ComponentTaskHandler → facade resolves the Gmail connection bound to this deployment → the component's perform runs with context.http(...).
5. Output goes to file storage; TaskExecutionCompleteEvent returns to the coordinator; the output is merged into context as searchEmail_1; the loop dispatcher takes over for the next task; and so on until the job completes.

Multi-tenancy

A single ByteChef installation can serve one tenant or many. BYTECHEF_TENANT_MODE selects which (SINGLE, the default, or MULTI); multi-tenant mode is an Enterprise Edition capability.

Isolation is schema-per-tenant, not database-per-tenant. There is one PostgreSQL database and one connection pool. Every tenant gets its own schema named bytechef_<tenantId>, and a DataSource wrapper issues SET search_path TO <schema> on each connection it hands out, based on the tenant bound to the current context. The practical guarantee is the same one a separate database would give for query isolation — a query for "workflows" in tenant A cannot return tenant B's rows, because they are not in the same table — without the connection-pool multiplication that per-tenant pools imply.

Consequences worth planning for:

• Migrations run per tenant. Liquibase is applied once per tenant schema at startup, so the migration window grows with the number of tenants.
• The scheduler is tenant-aware, not tenant-partitioned. There is one shared Quartz job store. Schedule and polling trigger keys are built from a WorkflowExecutionId that carries the tenant id, and connection-refresh keys are prefixed with it, so those are addressable and cancellable per tenant — but they are not stored separately. The prefixing is not universal: one-time tasks (the delayed wake-ups behind a suspended run) are keyed on the bare job id, with no tenant component.
• Internal service calls carry the tenant. Where one ByteChef process calls another over /remote/**, the tenant id travels in a CURRENT_TENANT_ID header alongside the internal service token, and the receiver validates both.
• No cross-tenant queries. The model assumes you never join across tenants; cross-tenant analytics belong in a warehouse fed by per-tenant exports.
• The tenant id reaches the logs. TenantContext puts tenantId into the SLF4J MDC, so it is available for filtering in every log line.

Encryption is not per tenant: the platform holds one instance-wide encryption key (see Encryption of stored credentials). Tenant isolation is enforced by the schema boundary and the request-scoped tenant context, not by separate key material.

Deployment shapes

The same binary set runs in two shapes, and what stays identical across them is the point: the database schema, the component model, the workflow definition, and the observability pipeline. A component definition and a workflow JSON behave the same either way.

Notifications and alerts

The engine itself is notification-agnostic — nothing under server/libs/atlas/ sends email or webhooks. Instead, every job status transition publishes a JobStatusApplicationEvent, and the coordinator's listener fan-out (in platform-coordinator) turns those events into deliveries. platform-notification is the central registry for notification channels (EMAIL, WEBHOOK, SLACK); other features reference Notification rows as delivery targets instead of defining their own channel entities.

Alert rules (Enterprise) are workspace-scoped rows that own the when — consecutive failures, failure rate, error count, latency threshold/spike, cost threshold, usage threshold, no activity — while the notification registry owns the where and how. Rule evaluation state lives on the rule row and is updated per terminal job event; time-based rules (no-activity, usage-threshold) fire from scheduled monitors.

Related reading

• Build a component — the hands-on authoring guide
• Component specification — every DSL method, one by one
• Working with triggers — trigger authoring specifics
• AI Agent — the cluster-root component in depth, including cluster-element slots
• Cloud — run this architecture without managing any of it yourself
• Self-hosting configuration — broker/worker/scheduler properties