How does an Nx self-hosted remote cache work?
Every cacheable Nx task, such as build, test or lint for a given project, gets a computation hash before it runs. Nx derives that hash from the task's declared inputs: the project's source files as defined by its inputs configuration, the hashes of the projects it depends on, relevant root config, the target's command and flags, and any environment variables or runtime values you have explicitly declared. If any of these change, the hash changes.
When the task finishes, Nx stores two things under that hash: the files matched by the target's outputs and the terminal log. On a later run with the same hash, Nx restores the files and replays the log instead of executing the task. It checks the local cache first. With a remote cache configured, it also checks a shared store.
The backing store is just a bucket
A self-hosted remote cache is an object storage bucket where each hash maps to an archived artifact. CI runners and developer machines all read from and write to the same bucket. If one CI job builds a library, the next job and every developer who pulls that commit can reuse the result without rebuilding it. Teams adopt it for this reason: work that already ran on one machine doesn't have to run again on another, which cuts CI minutes and the time engineers spend waiting.
The correctness dependency
The cache can only be as accurate as your input declarations. If a build reads an environment variable or a file that the target's inputs don't list, two builds with different results can produce the same hash. Nx will then report a hit and restore the wrong output. With a local cache, this affects one machine. With a remote cache, the bad artifact reaches everyone who requests that hash.
+------------------------+ +------------------------+
| Machine A: CI runner | | Machine B: CI or dev |
| | | |
| declared inputs: | | same inputs => same H |
| - source files | | |
| - dep project hashes | | 1. check local cache |
| - root config | | 2. check remote bucket |
| - command + flags | | 3. hit: restore outputs|
| - declared env vars | | and replay the log |
| | | | |
| v | +-----------^------------+
| hash H = f(inputs) | |
| | | |
| v | | GET H
| miss: run the task | |
| | | |
| v | |
| PUT H: outputs + log | |
+----------|-------------+ |
v |
+---------------------------------------------+------------+
| Remote cache: object storage bucket (keyed by hash) |
| H1 -> archive of outputs + terminal log |
| H2 -> archive of outputs + terminal log |
| shared by every CI runner and developer machine |
+----------------------------------------------------------+
Undeclared env var or file read => same H, different output
=> wrong archive is restored for everyone who requests H
Nx hashes a task's declared inputs, stores outputs and log under that hash in a shared bucket, and restores them on any machine that computes the same hash.
Where do the S3 and GCS cache plugins fit in Nx Powerpack?
The S3 and GCS cache plugins are part of Nx Powerpack, a set of add-on packages that extend the open-source Nx CLI. Each plugin implements the remote cache interface against one storage backend. Nx still computes task hashes and decides hit or miss; the plugin only reads and writes artifacts in your bucket. Because the bucket is yours, the poisoned-artifact scenario above becomes your problem to prevent and detect.
Powerpack's licensing, activation steps, and plugin lineup have changed across Nx releases, including package names and how a license is supplied. Check the official Nx documentation for your Nx version before installing. Pin the plugin to an exact version that matches your Nx version, and recheck the docs on every Nx upgrade. A mismatched plugin can fail quietly and fall back to local-only behavior, so the only symptom may be slower CI.
Alternatives and what each makes you own
| Option | Who runs storage | What you own |
|---|---|---|
| Nx managed remote caching (Nx Cloud) | Nx | Access tokens, workspace configuration |
| S3 or GCS plugin | You, in your cloud account | Encryption, access logging, retention and lifecycle rules, IAM, incident response |
| Custom cache server | You, end to end | All of the above, plus the server's code, uptime, and API compatibility |
| Local-only cache | Each machine | Nothing shared, and no cross-machine reuse |
Self-hosting suits teams that must keep build artifacts inside their own cloud boundary and already run IAM, logging, and retention policies for other buckets.
How do you configure the Nx S3 cache and Nx GCS cache in nx.json?
Both plugins follow the same pattern. You install the cache package for your cloud, then add a small block to nx.json that tells Nx which bucket to use. Nothing in that block should be secret. The package names and top-level keys have changed across Nx releases, so confirm the exact names against the Nx docs for your installed version before copying anything.
What goes in nx.json
For S3, you supply the bucket name and the region. Region matters because S3 API calls are signed per region, and pointing at the wrong one causes redirects or signature errors. Some setups also need an endpoint override for S3-compatible storage. Check whether your version supports that.
For GCS, you usually need only the bucket name. Bucket names are globally unique, and the project is inferred from the credentials. Location is set when you create the bucket, not in Nx.
Where credentials come from
Neither block should hold access keys. Each plugin uses its cloud's standard SDK credential chain:
- On AWS: environment variables, then shared config and profiles, then instance or container metadata.
- On Google Cloud: Application Default Credentials, which can come from an environment variable pointing at credentials, gcloud user credentials, or the metadata server.
Keeping keys out of nx.json keeps them out of version control. It also means the same config works on a laptop and in CI.
| Step | S3 | GCS |
|---|---|---|
| Install | S3 cache package | GCS cache package |
| Required config | Bucket, region | Bucket |
| Optional config | Custom endpoint (verify support) | Usually none |
| Credential source | AWS SDK chain | Application Default Credentials |
| Local dev auth | Profile or SSO session | gcloud user login |
# Illustrative setup. Confirm package names and nx.json keys against the Nx docs for your installed version.
# --- S3 ---
npm install --save-dev @nx/s3-cache
# nx.json fragment: bucket and region only, no keys
cat <<'EOF'
{
"s3": {
"bucket": "acme-nx-cache",
"region": "eu-west-1"
}
}
EOF
# Local dev: let the AWS SDK credential chain find a named profile
aws sso login --profile acme-dev
export AWS_PROFILE=acme-dev
# --- GCS ---
npm install --save-dev @nx/gcs-cache
# nx.json fragment: bucket only. Project comes from credentials, location from bucket creation
cat <<'EOF'
{
"gcs": {
"bucket": "acme-nx-cache"
}
}
EOF
# Local dev: write Application Default Credentials for the SDK to pick up
gcloud auth application-default login
# In CI, set neither of the local-dev logins. Rely on instance or workload
# metadata, or on standard SDK environment variables injected by the runner.
npx nx run-many -t build
This illustrative setup shows the S3 and GCS cache configuration side by side: each nx.json fragment holds only non-secret bucket settings, and credentials come from the AWS SDK chain or Google Application Default Credentials.
How should CI authenticate to the cache bucket without long-lived keys?
Both clouds let CI exchange a short-lived identity token from the CI provider for temporary cloud credentials. Neither requires storing a secret in the pipeline. On AWS, the CI provider is registered as an OIDC identity provider in IAM. The job then calls AssumeRoleWithWebIdentity against a role whose trust policy restricts the token's sub claim to specific repositories and branches. On GCP, a workload identity pool trusts the same OIDC issuer, and the job impersonates a service account that has bucket permissions. Attribute conditions restrict which repositories can do this.
| Aspect | AWS | GCP |
|---|---|---|
| Trust anchor | IAM OIDC identity provider | Workload identity pool and provider |
| What CI receives | Temporary STS credentials | Short-lived service account token |
| Scoping lever | Role trust policy conditions on sub |
Attribute conditions on token claims |
| Long-lived fallback to avoid | IAM user access keys | Service account JSON keys |
Static keys are the riskier option for three reasons:
- They leak. Keys end up in CI logs, forked workflows, and
.envfiles. - They rarely get rotated. Rotating means updating every pipeline that holds the key.
- They are hard to scope. One key is usually shared across all branches, so a pull request job and a release job get the same write access.
On GitHub Actions, the job grants itself the id-token: write permission and then runs the cloud's official auth step before calling nx. A snippet with placeholder role and pool names follows this section.
Developer laptops are harder. AWS SSO sessions or gcloud auth application-default login both work. However, each engineer must be onboarded, and each one becomes another identity with bucket access. Count every CI role, laptop user, and break-glass account that can reach the bucket. That full list is the attack surface section 5 has to restrict.
name: ci
on: [push, pull_request]
# Allow jobs to request a GitHub OIDC token; no cloud secrets are stored.
permissions:
id-token: write
contents: read
jobs:
build-aws:
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@v4
# Exchange the OIDC token for temporary STS credentials.
# The role's trust policy should limit the token's sub claim to this repo and branch.
- uses: aws-actions/configure-aws-credentials@v4
with:
role-to-assume: arn:aws:iam::123456789012:role/nx-cache-ci
aws-region: us-east-1
- run: npm ci
- run: npx nx affected -t build test
build-gcp:
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@v4
# Impersonate a service account through workload identity federation.
# Attribute conditions on the provider decide which repos may do this.
- uses: google-github-actions/auth@v2
with:
workload_identity_provider: projects/123456789/locations/global/workloadIdentityPools/github-pool/providers/github-provider
service_account: nx-cache-ci@example-project.iam.gserviceaccount.com
- run: npm ci
- run: npx nx affected -t build test
Illustrative GitHub Actions workflow where each job gets short-lived cloud credentials through OIDC federation, an IAM role on AWS and service account impersonation on GCP, before running Nx, with no stored access keys or JSON keys.
How do you prevent cache poisoning with least-privilege bucket policies?
That list matters because a remote cache is trusted implicitly. When Nx gets a cache hit, it restores the stored outputs and skips the task. It does not rebuild them to check that they match. Any identity that can write an object under a hash that a protected build will later compute can plant a tampered dist/ folder, and that folder can flow into a release. A fork pull request is the obvious threat, because it runs attacker-controlled code. If that job holds write credentials, the attacker never needs to touch your main branch.
Split access by trust level:
| Identity | S3 permissions | GCS role binding | Rationale |
|---|---|---|---|
| CI on protected branches | s3:GetObject, s3:PutObject on arn:aws:s3:::bucket/* |
roles/storage.objectViewer + roles/storage.objectCreator |
Reviewed code produces the artifacts |
| CI for fork PRs | s3:GetObject only, or none |
roles/storage.objectViewer, or none |
Untrusted code must not write |
| Developer machines | s3:GetObject only, or none |
roles/storage.objectViewer, or none |
Laptops are easier to compromise than CI |
On GCS, objectCreator does not include delete permission, and overwriting an existing object requires it. Writers can add new entries but cannot replace existing ones. On S3, do not grant s3:DeleteObject or bucket-wide s3:*.
Both providers encrypt objects at rest by default. Use customer-managed keys if your policy requires control over key revocation. Turn on S3 server access logging or CloudTrail data events, or Cloud Storage Data Access audit logs, which are off by default. Without logs, you cannot tell which identity wrote a suspicious entry.
#!/usr/bin/env bash
set -euo pipefail
BUCKET=acme-nx-cache
# S3: protected-branch CI role may read and add cache entries.
# No s3:DeleteObject and no s3:*. Without s3:ListBucket, a cache miss
# returns 403 instead of 404.
cat > nx-cache-rw.json <<EOF
{
"Version": "2012-10-17",
"Statement": [{
"Effect": "Allow",
"Action": ["s3:GetObject", "s3:PutObject"],
"Resource": "arn:aws:s3:::${BUCKET}/*"
}]
}
EOF
aws iam put-role-policy --role-name ci-protected-branch \
--policy-name nx-cache-rw --policy-document file://nx-cache-rw.json
# Fork-PR CI role: same shape, but s3:GetObject only.
sed 's/, "s3:PutObject"//' nx-cache-rw.json > nx-cache-ro.json
aws iam put-role-policy --role-name ci-fork-pr \
--policy-name nx-cache-ro --policy-document file://nx-cache-ro.json
# GCS: the trusted CI service account can read and create objects.
# objectCreator has no delete permission, so it cannot overwrite existing entries.
GCS_BUCKET=gs://acme-nx-cache
CI_SA=serviceAccount:nx-ci-main@example-project.iam.gserviceaccount.com
RO_SA=serviceAccount:nx-ci-forks@example-project.iam.gserviceaccount.com
for role in roles/storage.objectViewer roles/storage.objectCreator; do
gcloud storage buckets add-iam-policy-binding "$GCS_BUCKET" \
--member="$CI_SA" --role="$role"
done
# The fork-PR service account gets read-only access.
gcloud storage buckets add-iam-policy-binding "$GCS_BUCKET" \
--member="$RO_SA" --role=roles/storage.objectViewer
Grants least-privilege Nx cache access: read-write for protected-branch CI and read-only for fork PRs, using an inline S3 IAM policy and GCS bucket role bindings.
What does a self-hosted Nx cache cost, and when should you avoid it?
Neither S3 nor GCS evicts old objects for you. Every task hash Nx writes stays in the bucket until something deletes it, so a bucket without lifecycle rules grows without limit.
| S3 | GCS | |
|---|---|---|
| Mechanism | Lifecycle configuration, Expiration action | Object Lifecycle Management, Delete action |
| Age basis | Days since object creation | age condition (days since creation) |
| Scope | Prefix or tag filter | Prefix/suffix match conditions |
Both rules count age from when the object was written, not when it was last read. Nx writes each hash once, so a 14-day rule deletes an artifact that a release branch still hits every day, and that branch rebuilds from scratch. Short expiry lowers cost but cuts hit rates on long-lived branches. If you need both, write release-branch artifacts under a separate prefix and give that prefix a longer rule.
Illustrative estimate (assumed numbers, not a benchmark). Assume 200 GB stored, 2,000,000 GETs, 300,000 PUTs, and 3,000 GB downloaded each month by runners in another region.
Monthly cost = (200 × STORAGE_RATE) + (2,000 × GET_RATE_PER_1K) + (300 × PUT_RATE_PER_1K) + (3,000 × EGRESS_RATE)
Replace each placeholder with the current list price. Egress here is 15× the stored volume, and per-GB transfer rates are usually higher than per-GB-month storage rates. That means egress and cross-region traffic usually outweigh storage cost.
Avoid self-hosting, or change the layout, in three cases:
- Your runners sit in a different cloud or a distant region.
- Your artifacts are large enough that a fast runner rebuilds them quicker than it can download them.
- No one on the team has the capacity to own the bucket's policies, logs, and lifecycle rules.
How do you verify the cache works and choose between S3 and GCS?
Run the same target three ways on a clean CI runner. First, run it with --skip-nx-cache to get a true uncached baseline. Second, run it normally so the remote cache is populated. Third, run nx reset to clear the local cache and run it again. Only that third run proves artifacts came from the bucket and not from disk. In the CI log, look for tasks tagged as read from the remote cache and for the summary line reporting how many tasks were restored from cache. If the third run rebuilds everything, the hash inputs differ between runs, usually because of an undeclared environment variable or a generated file.
Then test the negative paths, because a cache that accepts everything is the real risk:
- Read access with an identity that has no grant should fail with an access-denied error.
- A workflow triggered from a fork PR should be able to read but must be refused on write. Confirm the refusal appears in the bucket's audit logs, not only in the CI output.
| Decision factor | Choose S3 when | Choose GCS when |
|---|---|---|
| CI identity | Runners assume AWS roles via OIDC | Runners use Workload Identity Federation |
| Runner location | Runners run in an AWS region | Runners run in a GCP region |
| Existing audit tooling | CloudTrail is already monitored | Cloud Audit Logs are already monitored |
Before committing to either, check these:
- Put the bucket in the same region as the runners. This avoids cross-region egress fees and download latency.
- Make sure every hash input is declared. Otherwise cache misses will look like a storage problem.
- Compare the baseline and remote-hit durations over a week of real pipelines before you treat the cache as a saving.
The two plugins behave almost the same. The choice should come down to how your CI identity and network paths are set up.
How OneEnv can help
Disclosure: OneEnv is the product behind this blog.
OneEnv (oneenv.io) is a workspace-based governance platform for the things services share — environment variables, database structure, API routes and validation rules. A change to a shared value is staged on a branch, reviewed once, and then every affected service explicitly approves or rejects it before it goes live, so you can see who a change touches before it ships instead of finding out in production.
Applied to nx self-hosted remote cache, this is where it helps:
Saves time
- Every service that inherits a shared value is listed in one place, so nobody searches repos and chats to find out who is affected.
- One review per branch replaces coordinating each team separately; each service still keeps its own approve/reject.
- Reviewers see the exact diff and the list of affected services up front, so approval is a decision rather than an investigation.
Saves cost
- Fewer production incidents and rollbacks caused by a shared value, route or column changing under a consumer unannounced.
- Less duplicated configuration to maintain: one shared definition inherited by many services.
- Per-service approve/reject means a bad shared value can be refused where it would break something, before it is live.
- OneEnv has a free plan (1 workspace, unlimited services and members, full governance), so trying it costs nothing.
These are qualitative benefits, not measured results — how much you save depends on how many services share the value.
Want to try it? Start at oneenv.io.