How NAT Gateway billing works and where the money goes
A NAT Gateway bill has three parts, and they scale with different things.
Hourly charge. Each NAT Gateway is billed for every hour it exists, whether or not traffic flows through it. The resilient pattern is one gateway per Availability Zone, so a three-AZ VPC pays this charge three times, and every additional VPC built from the same template multiplies it again. This cost is fixed and predictable, and it is rarely where the surprise comes from.
Per-GB data processing. Every gigabyte that passes through the gateway is charged, in both directions, regardless of destination. Traffic to an S3 bucket or an ECR registry in the same region is charged at the same rate as traffic to an external API. Internet egress is billed separately on top of this. This is the line that grows with workload volume.
Cross-AZ data transfer. If a subnet's route table points to a NAT Gateway in a different AZ, which happens when a gateway was removed or a template hardcoded a single one, each byte also incurs inter-AZ transfer charges before it reaches the gateway.
Most avoidable spend is the processing charge on traffic to AWS's own services. Container image pulls, S3 reads and writes for logs, artifacts and data pipelines, and calls to CloudWatch, STS and Secrets Manager all leave private subnets through the NAT Gateway by default. None of that traffic needs to reach the internet, yet it is billed as if it did. This matters most to platform and backend teams running EKS, ECS or batch workloads in private subnets, where data volume is high and the cost is easy to overlook.
+---------------------------- VPC (one region) ----------------------------+
| |
| +------------ AZ-a -------------+ +------------ AZ-b -------------+ |
| | Private subnet | | Private subnet | |
| | EKS / ECS / batch | | EKS / ECS / batch | |
| +---------------+---------------+ +---------------+---------------+ |
| | | |
| | [3] cross-AZ data transfer | |
| |<----------------------------------+ |
| v route points to AZ-a NAT |
| +-------------------------------+ |
| | NAT Gateway (AZ-a) | [1] hourly, per gateway, per AZ |
| | [2] per-GB processing, both | |
| | directions, any target | |
| +---------------+---------------+ |
| | |
+------------------+-------------------------------------------------------+
|
+----------+----------------------------+
v v
+--------------------------------+ +----------------------------------+
| AWS services, same region | | Internet / external APIs |
| S3, ECR, CloudWatch, STS, | | [2] processing charge |
| Secrets Manager | | + internet egress (separate) |
| [2] still charged per GB | +----------------------------------+
| = most avoidable spend |
+--------------------------------+
The three NAT Gateway charges: [1] an hourly fee per gateway per AZ, [2] per-GB processing on all traffic including same-region AWS service calls, and [3] cross-AZ transfer when a subnet routes to a NAT Gateway in another AZ.
Measure NAT traffic before changing anything
Before adding endpoints, find out which services the NAT bytes go to. Endpoints have their own costs and operational overhead. A short measurement pass shows which ones will pay off and which would carry almost no traffic.
Split the bill first
In Cost Explorer, group by usage type and filter for NAT. You will see two lines, each prefixed with a region code: NatGateway-Hours and NatGateway-Bytes. Endpoints only reduce the bytes line. If hourly charges dominate, consolidating NAT Gateways matters more than adding endpoints.
Attribute bytes with Flow Logs
Enable VPC Flow Logs on the NAT Gateway's network interface. Use a custom format that includes pkt-srcaddr and pkt-dstaddr. Without these fields, the destination address only shows the NAT's private IP, not the remote service. Send the logs to S3 and query them with Athena. Sum bytes grouped by pkt-dstaddr, filter to accepted traffic, and sort descending. A day of data is usually enough to see the pattern.
Map IPs to services
AWS publishes its address ranges as ip-ranges.json. A small script does the matching:
- Load the prefixes.
- For each top destination, find the most specific matching prefix and record its
serviceandregion. - Total the bytes per service.
S3 has its own label. ECR layer downloads are served from S3, so they appear under S3. ECR API calls, CloudWatch and STS usually appear only as the generic AMAZON label. To tell them apart, use Route 53 Resolver query logs or reverse lookups.
# Usage: python3 -I map_nat_bytes.py ip-ranges.json top_destinations.csv
# The CSV is the Athena result: columns dst,bytes, grouped by pkt-dstaddr.
# IPv4 only. Add data["ipv6_prefixes"] if your NAT carries IPv6.
import csv, ipaddress, json, sys
from collections import defaultdict
ranges_path, csv_path = sys.argv[1], sys.argv[2]
with open(ranges_path) as f:
data = json.load(f)
prefixes = [(ipaddress.ip_network(p["ip_prefix"]), p["service"], p["region"])
for p in data["prefixes"]]
def classify(ip):
addr = ipaddress.ip_address(ip)
best = None
for net, service, region in prefixes:
if addr in net:
# Longest prefix wins. On a tie, prefer a specific label (S3, EC2...)
# over the generic AMAZON entry that overlaps most ranges.
key = (net.prefixlen, service != "AMAZON")
if best is None or key > best[0]:
best = (key, service, region)
return (best[1], best[2]) if best else ("NON-AWS", "-")
# A linear scan is fine here because the input is only the top-N destinations.
totals = defaultdict(int)
with open(csv_path, newline="") as f:
for row in csv.DictReader(f):
totals[classify(row["dst"])] += int(row["bytes"])
grand = sum(totals.values()) or 1
for (service, region), b in sorted(totals.items(), key=lambda kv: kv[1], reverse=True):
print(f"{service:<10} {region:<15} {b / 1e9:10.2f} GB {100 * b / grand:5.1f}%")
A Python script that matches the top NAT destination IPs from the Athena query against AWS's ip-ranges.json, using longest-prefix matching, and totals bytes per service and region so you can see which endpoints would remove meaningful NAT traffic.
Gateway endpoints for S3 and DynamoDB: add them everywhere
A gateway endpoint is not a network interface. Creating one adds a route to each route table you associate with it. The destination is an AWS-managed prefix list (pl-…) that holds the regional S3 or DynamoDB address ranges, and the target is the endpoint ID. That route is more specific than the 0.0.0.0/0 route to the NAT gateway, so matching traffic skips the NAT entirely. Gateway endpoints have no hourly charge and no per-GB charge. Every private subnet route table should be associated with one. Leaving a table out saves nothing and keeps S3 traffic, including the ECR layer downloads from the previous section, on the NAT gateway's processing charge.
The default endpoint policy allows full access. To restrict it, write a policy that permits s3:GetObject and s3:PutObject only on your buckets, as in the example that follows. If you scope it this way, also allow the AWS-owned buckets your workloads depend on. The regional ECR layer bucket is the common one to forget, and when it is missing, image pulls fail with access denied errors rather than falling back to the NAT.
Know the limits before rollout:
- Same region only. Requests to buckets in other regions still go through the NAT gateway.
- S3 and DynamoDB only. Every other service requires an interface endpoint.
- Source IP changes. Requests arrive from private addresses rather than the NAT's Elastic IP. Any bucket policy that allows the NAT EIP through
aws:SourceIpwill start denying those requests. Rewrite those conditions to useaws:SourceVpceoraws:SourceVpcbefore you associate the route tables.
#!/usr/bin/env bash
# Create an S3 gateway endpoint with a restrictive policy and attach it
# to every private route table in one call.
set -euo pipefail
REGION=us-east-1
VPC_ID=vpc-0abc1234def567890
# Every private subnet route table. Any table left out keeps sending S3 traffic through the NAT.
ROUTE_TABLES="rtb-0aaa1111bbbb2222c rtb-0ddd3333eeee4444f"
cat > s3-endpoint-policy.json <<'EOF'
{
"Version": "2012-10-17",
"Statement": [
{
"Sid": "AppBucketsOnly",
"Effect": "Allow",
"Principal": "*",
"Action": ["s3:GetObject", "s3:PutObject"],
"Resource": ["arn:aws:s3:::example-app-data/*", "arn:aws:s3:::example-app-logs/*"]
},
{
"Sid": "EcrImageLayers",
"Effect": "Allow",
"Principal": "*",
"Action": "s3:GetObject",
"Resource": "arn:aws:s3:::prod-us-east-1-starport-layer-bucket/*"
}
]
}
EOF
# Adds a route with the S3 prefix list (pl-...) as destination and the endpoint as target.
# No hourly or per-GB charge applies to gateway endpoints.
aws ec2 create-vpc-endpoint \
--region "$REGION" \
--vpc-id "$VPC_ID" \
--vpc-endpoint-type Gateway \
--service-name "com.amazonaws.${REGION}.s3" \
--route-table-ids $ROUTE_TABLES \
--policy-document file://s3-endpoint-policy.json
Creating an S3 gateway endpoint with the AWS CLI, associating it with all private route tables, and limiting it to the application buckets plus the regional ECR layer bucket so image pulls keep working.
Why ECR pulls dominate NAT spend and how to take them off NAT
Container images look small in a registry UI, but every node that lacks a cached copy downloads the full set of compressed layers. ECR does not serve those layers itself. The ecr.dkr endpoint returns the manifest and a redirect, and the layer bytes come from S3. A fully private pull path therefore needs three pieces:
- the S3 gateway endpoint
- the
ecr.apiinterface endpoint - the
ecr.dkrinterface endpoint
If you add only the two ECR interface endpoints, the layer traffic, which is the bulk of the volume, still goes through the NAT gateway.
Several things multiply pull volume. Autoscaling adds nodes with empty image caches. Each deploy pushes a new tag that every node running the workload must fetch. Large base images make each fetch bigger. Short-lived nodes, such as spot instances or aggressive scale-in, discard their caches before they can be reused.
Illustrative estimate (assumed numbers, not a benchmark)
Monthly NAT cost from pulls = deploys per day × nodes pulling per deploy × image size in GB × 30 × NAT per-GB processing rate
With 20 deploys per day, 30 nodes, a 1.5 GB image and an assumed rate of $0.045/GB, the cost is 20 × 30 × 1.5 × 30 × $0.045 = $1,215/month. That figure excludes pulls caused by scale-out events.
If your S3 endpoint policy restricts access to your own buckets, it must also allow reads from the regional bucket that ECR uses for layer storage. If it doesn't, manifests resolve but layer downloads fail, typically showing up as access-denied or timeout errors during the pull, which look like registry or credential problems.
Endpoints do not help with third-party registries. Pulls from Docker Hub and other public registries still go through the NAT gateway.
#!/usr/bin/env bash
# Private ECR pull path: S3 gateway endpoint + ecr.api + ecr.dkr interface endpoints.
set -euo pipefail
REGION=us-east-1
VPC_ID=vpc-0abc1234def567890
RTB_IDS="rtb-0aaa1111bbbb2222c rtb-0ddd3333eeee4444f" # private subnet route tables
SUBNET_IDS="subnet-0123abcd4567ef890 subnet-0fedc9876ba543210"
SG_ID=sg-0123456789abcdef0 # must allow HTTPS (443) inbound from the nodes
# The S3 policy allows your own bucket AND the regional bucket where ECR stores layers.
# Without the second statement, manifests resolve but layer downloads fail.
cat > s3-endpoint-policy.json <<EOF
{
"Version": "2012-10-17",
"Statement": [
{ "Sid": "AppBucket", "Effect": "Allow", "Principal": "*",
"Action": ["s3:GetObject", "s3:PutObject", "s3:ListBucket"],
"Resource": ["arn:aws:s3:::example-app-bucket", "arn:aws:s3:::example-app-bucket/*"] },
{ "Sid": "EcrLayers", "Effect": "Allow", "Principal": "*",
"Action": "s3:GetObject",
"Resource": "arn:aws:s3:::prod-${REGION}-starport-layer-bucket/*" }
]
}
EOF
# Gateway endpoint: layer bytes go here instead of through the NAT gateway.
aws ec2 create-vpc-endpoint --region "$REGION" --vpc-id "$VPC_ID" \
--vpc-endpoint-type Gateway --service-name "com.amazonaws.${REGION}.s3" \
--route-table-ids $RTB_IDS --policy-document file://s3-endpoint-policy.json
# Interface endpoints for the registry API and the Docker registry protocol (manifests).
for svc in ecr.api ecr.dkr; do
aws ec2 create-vpc-endpoint --region "$REGION" --vpc-id "$VPC_ID" \
--vpc-endpoint-type Interface --service-name "com.amazonaws.${REGION}.${svc}" \
--subnet-ids $SUBNET_IDS --security-group-ids "$SG_ID" --private-dns-enabled
done
Creates the S3 gateway endpoint plus the ecr.api and ecr.dkr interface endpoints, with an S3 endpoint policy that restricts access to your own bucket while still allowing reads from the regional ECR layer bucket so image pulls keep working.
Interface endpoints and the break-even calculation
Interface endpoints use PrivateLink: each one places an elastic network interface in every subnet you select. You pay an hourly charge for each endpoint in each AZ, plus a per-GB processing fee that is lower than NAT's. Each endpoint covers a single service, so the hourly charge is the real cost. ECR alone needs two endpoints (ecr.api and ecr.dkr).
Common candidates are ECR, CloudWatch Logs, STS, Secrets Manager, SSM, KMS and SQS. Two settings decide whether traffic actually uses the endpoint:
- Private DNS enabled: the service's default hostname then resolves to the endpoint's private IPs, so SDKs and agents use it without configuration changes.
- A security group allowing inbound 443 from your workloads: without this rule, connections time out instead of falling back to NAT.
Break-even
An endpoint pays for itself when:
hourly endpoint price × AZs × hours per month < monthly GB × (NAT per-GB rate − endpoint per-GB rate)
Illustrative estimate. Assumed inputs: an endpoint at $0.01/hour, 3 AZs, 730 hours per month, NAT processing at $0.045/GB and endpoint processing at $0.01/GB. The fixed cost is $21.90 per month per endpoint. Each GB moved saves $0.035, so break-even is about 626 GB per month. Low-volume services such as STS rarely reach that figure. Busy ECR or CloudWatch Logs traffic often does. Prices vary by region, so recalculate with current rates.
| Gateway endpoint | Interface endpoint | NAT Gateway | |
|---|---|---|---|
| Services | S3, DynamoDB | Many AWS services | Any destination |
| Hourly charge | None | Per endpoint per AZ | Per gateway |
| Per-GB charge | None | Lower than NAT | Highest |
| Mechanism | Route table entry | ENI plus DNS | Route table entry |
| Security group | Not applicable | Required (443) | Not applicable |
When to avoid interface endpoints and other trade-offs
Interface endpoints are billed per endpoint, per AZ, per hour, whether or not traffic flows. A dev VPC that pulls a few gigabytes a month will pay more for idle ENIs than it ever paid NAT for that traffic. The cost also multiplies: ten services across three AZs in twenty VPCs is six hundred hourly line items. Run the break-even per VPC, not once for the whole organization.
DNS conflicts
Private DNS works by overriding the public service hostname inside the VPC. That breaks down when instances use custom DNS servers instead of the VPC resolver, when on-premises resolvers forward queries in through hybrid DNS, or when an existing private hosted zone already claims the same name. In these cases clients either bypass the endpoint silently or resolve to addresses they cannot reach. Test resolution from a real workload, not just from the console.
Centralizing endpoints
Placing endpoints in a shared services VPC and routing other VPCs to it through Transit Gateway or peering cuts the hourly count, but adds costs and risks:
- Transit Gateway charges per GB processed, which can cancel out the savings.
- Spoke VPCs need the endpoint names shared with them, usually through private hosted zones associated across accounts or resolver rules.
- One misconfigured security group or route table cuts off AWS API access for every connected account.
NAT rarely disappears
OS package mirrors, third-party APIs, and SaaS monitoring or security agents still need general internet egress. Expect to shrink NAT traffic, not remove the gateway.
Alternatives that cut NAT traffic without endpoints
The remaining internet-bound traffic can still be reduced by sending fewer bytes or by routing them more cheaply.
One NAT Gateway per AZ. If private subnets in several AZs share a single NAT Gateway, traffic from the other AZs pays inter-AZ data transfer on top of NAT processing. Give each AZ its own gateway and point that AZ's route table at it. You pay an extra hourly charge per gateway, but you remove the cross-AZ charge and an AZ-wide single point of failure.
Smaller images and node-local caching. Each node pull is billed per byte. Use slim base images and multi-stage builds, and order layers so the large, stable ones stay cached. Pre-baked AMIs that already hold common images, or longer-lived nodes, mean fewer pulls, especially during scale-out.
Quieter agents and batched telemetry. Monitoring and security agents that poll every few seconds, or ship uncompressed logs line by line, add steady egress. Raise poll intervals, enable compression where the agent supports it, and batch before sending. Each request also carries TLS and HTTP header overhead, so fewer, larger requests cost less than many small ones.
| Option | Effort | What it cuts | Ongoing cost |
|---|---|---|---|
| NAT per AZ + route tables | Low | Cross-AZ transfer | Extra gateway-hours |
| Smaller images | Medium, per service | Pull bytes | None |
| Node-local image caching | Medium | Repeat pulls | AMI build pipeline |
| Tuning agents and telemetry | Low to medium | Steady egress | None |
| Interface endpoints | Low per service | Specific AWS APIs | Hourly per AZ |
Start with whichever row matches your largest traffic category from the measurement step.
Verify the change and track cost over time
Record a baseline before you deploy endpoints. In CloudWatch, chart BytesOutToDestination and BytesInFromDestination (namespace AWS/NATGateway) per NAT Gateway, summed daily, for at least a week. After the change, compare the same weekdays so batch jobs and deploy cycles don't skew the comparison. A drop in bytes is the first signal. Then rerun the flow-log attribution query from section 2 to confirm which traffic left the NAT path. Shrinking totals can hide a new top talker.
Confirm the routing, not only the config
For interface endpoints with private DNS enabled, run the check from an instance or pod inside the VPC, not from your laptop. Run dig +short api.ecr.<region>.amazonaws.com and dig +short <account-id>.dkr.ecr.<region>.amazonaws.com, or use nslookup if dig isn't installed. Both should return addresses from your subnet CIDRs. A public IP means the SDK still goes through NAT, usually because private DNS is off or the VPC uses a custom resolver.
S3 and DynamoDB gateway endpoints don't change DNS. Those hostnames still resolve to public IPs. To verify them, check that each private subnet's route table contains the service prefix-list route (aws ec2 describe-route-tables).
Keep watching
Set a CloudWatch alarm on NAT bytes against your new baseline. A new service calling an API you have no endpoint for, or a subnet associated with the wrong route table, then shows up within days instead of on next month's invoice. That saves the time an engineer would otherwise spend reconstructing the cause from billing data.
Finally, revisit the break-even calculation whenever AWS changes NAT or endpoint pricing, or when you add AZs:
savings = (GB moved off NAT × NAT per-GB rate) − (endpoints × AZs × hours × hourly rate + endpoint GB × endpoint per-GB rate)
How OneEnv can help
Disclosure: OneEnv is the product behind this blog.
OneEnv (oneenv.io) is a workspace-based governance platform for the things services share — environment variables, database structure, API routes and validation rules. A change to a shared value is staged on a branch, reviewed once, and then every affected service explicitly approves or rejects it before it goes live, so you can see who a change touches before it ships instead of finding out in production.
Applied to reduce nat gateway costs, this is where it helps:
Saves time
- Every service that inherits a shared value is listed in one place, so nobody searches repos and chats to find out who is affected.
- One review per branch replaces coordinating each team separately; each service still keeps its own approve/reject.
- Reviewers see the exact diff and the list of affected services up front, so approval is a decision rather than an investigation.
Saves cost
- Fewer production incidents and rollbacks caused by a shared value, route or column changing under a consumer unannounced.
- Less duplicated configuration to maintain: one shared definition inherited by many services.
- Per-service approve/reject means a bad shared value can be refused where it would break something, before it is live.
- OneEnv has a free plan (1 workspace, unlimited services and members, full governance), so trying it costs nothing.
These are qualitative benefits, not measured results — how much you save depends on how many services share the value.
Want to try it? Start at oneenv.io.