Skip to content

Private Worker Setup

A private worker runs inside your own network and connects out to Zenfra. Runs execute on your infrastructure; with worker-mode AWS integrations and customer-managed state, your cloud credentials and your Terraform state never leave it. Nothing needs to accept inbound connections: the worker dials out over HTTPS, claims runs for its pool, and executes Terraform or OpenTofu directly on the host it runs on.

This guide covers the Docker deployment. For what a pool is and how pools are managed, see Worker Pools.

Before you start

You need a machine with Docker and outbound HTTPS access to api.zenfra.cloud. The worker executes Terraform directly on the host, so run it on a machine dedicated to that purpose.

Create a worker pool first (Workers > Create Pool; requires the Admin role). The pool gives you the two values the worker needs: its Pool ID, and an API key shown once at creation. Store the key securely. If you lose it, rotate it from the pool's Actions menu, which issues a new one; the old key cannot be retrieved and every worker using it disconnects.

Run it

docker run -d --restart=unless-stopped --name zenfra-worker \
  -e ZENFRA_WORKER_API_BASE_URL=https://api.zenfra.cloud \
  -e ZENFRA_WORKER_POOL_ID=<pool_id> \
  -e ZENFRA_WORKER_POOL_API_KEY=<api_key> \
  ghcr.io/zenfracloud/zenfra-worker:latest

Replace <pool_id> and <api_key> with the values from Zenfra. The pool detail page shows this command with your Pool ID already filled in.

The container runs detached and restarts with the Docker daemon. Within a few seconds it registers and appears under Connected Workers on the pool page. If the API is unreachable it keeps retrying for up to two minutes before exiting, so give it a moment before assuming something is wrong. If it does not appear, docker logs zenfra-worker says why: configuration problems are reported by name and the container exits rather than running in a broken state.

These three variables are the only required configuration.

Variable Value
ZENFRA_WORKER_API_BASE_URL https://api.zenfra.cloud, with no trailing slash
ZENFRA_WORKER_POOL_ID The Pool ID from the pool detail page
ZENFRA_WORKER_POOL_API_KEY The API key shown once when the pool was created or the key was rotated

Optional: persist workspaces and the tool cache

Add these two mounts to the command to keep state across restarts:

  -v /opt/zenfra-worker/workspaces:/workspace \
  -v /opt/zenfra-worker/tools-cache:/var/cache/zenfra/tools \

Without them the worker still works, but each time the container is replaced (as in Upgrading below) it re-downloads the Terraform and OpenTofu binaries it had already fetched, and any workspace left behind by an interrupted run is lost. On a machine that runs frequently, the tool cache is the one worth having.

Run logs and disk space

Tool output is spooled to disk on the worker and uploaded independently, so a slow or unreachable API delays the upload but never blocks the apply. Zenfra then checks that every byte reached it. If any output was lost, for example because the spool ran out of room, the run is marked Failed with log_upload_incomplete even when the apply itself succeeded. Check your cloud before re-running such a run.

The spool lives under the workspace directory (/workspace/spool by default, so it lands on the workspaces mount above if you use it). Defaults:

Environment variable YAML key Default
ZENFRA_LOGS_SPOOL_DIR logs.spool_dir /workspace/spool
ZENFRA_LOGS_SPOOL_QUOTA_PER_RUN_BYTES logs.spool_quota_per_run_bytes 1 GiB
ZENFRA_LOGS_SPOOL_QUOTA_TOTAL_BYTES logs.spool_quota_total_bytes 8 GiB
ZENFRA_LOGS_SPOOL_FREE_SPACE_FLOOR_BYTES logs.spool_free_space_floor_bytes 512 MiB

Spooling stops when free space on the filesystem holding the spool directory drops below the floor. Size that filesystem (by default the one behind /workspace) for the workspaces plus the total spool quota plus the floor. Spool files are removed when the run's log stream closes or the workspace is cleaned up.

The YAML keys apply if you run the worker with a config file (ZENFRA_WORKER_CONFIG=<path> or -config <path>); environment variables override it.

Image tags and versions

The image is ghcr.io/zenfracloud/zenfra-worker. Only the tags latest, main and main-<sha> are published; there are no version tags to pin to. latest and main point at the same build.

To see which build you are running:

docker run --rm ghcr.io/zenfracloud/zenfra-worker:latest --version

prints zenfra-worker <version> (<commit>). The worker also reports its version and commit when it registers, so support can see what you run.

AWS identity for worker-mode integrations

An AWS integration created with Generate credentials in worker makes the worker assume your IAM role itself, with whatever AWS identity the worker process has; Zenfra's own role is not in the trust chain. Such an integration can only be used by stacks on a private pool. Give the worker an identity through the AWS SDK's default credential chain: an EC2 instance profile, IRSA or EKS Pod Identity on Kubernetes, or environment variables on the container. The AWS integration page describes the permissions the identity needs and how the trust policy must name it.

If the worker runs in Docker on EC2 with an instance profile, raise the IMDSv2 hop limit to 2 so the container can reach the instance metadata service:

aws ec2 modify-instance-metadata-options --instance-id <id> --http-endpoint enabled --http-put-response-hop-limit 2

Upgrading

Docker does not re-pull an image on restart, and docker restart re-runs the container you already have. Upgrading is three steps:

docker pull ghcr.io/zenfracloud/zenfra-worker:latest
docker rm -f zenfra-worker
# then run the command above again

Environment variables are fixed when a container is created, so this is also how you change any setting: a restart will not pick up a new pool ID or API key.

Finish any in-flight runs before removing the container. A run that was executing when its worker disappeared is not resumed or retried. It stays active until its deadline passes, then the platform marks it failed and it must be started again.

Troubleshooting

Read docker logs zenfra-worker. The common causes report themselves plainly:

  • ZENFRA_WORKER_API_BASE_URL is required: the variable is missing, or was set to an empty value. If it came from a shell substitution, check the source exists.
  • must not have trailing slash: remove the trailing / from the URL.
  • ZENFRA_WORKER_POOL_API_KEY is required for private_api_key registration mode: the key is missing or misspelled. Note it is POOL_API_KEY, not API_KEY.
  • The container starts and immediately exits with no message: usually the image failed to pull. Run docker pull on its own to see the registry error.
  • A run fails with worker credential mode requires a private worker pool: the stack uses an AWS integration with Generate credentials in worker enabled but runs on the Public Worker Pool. Move the stack to a private pool, or switch the integration to control-plane mode.