What drives a CloudWatch Logs bill?

A CloudWatch Logs bill has three main line items, and each one responds to a different lever. This checklist is for backend and platform engineers who own log groups, or who answer for the account-level bill and need to know which changes will actually reduce it.

Cost component Billed per What drives it Main lever
Ingestion GB ingested Log volume your code and services emit Log less, or use a cheaper log class
Storage (archived) GB-month stored Volume multiplied by retention period Retention policy
Logs Insights queries GB scanned Query frequency, time range, number of log groups selected Narrower queries, fewer dashboards that auto-run

Retention is the most visible setting, and it matters because the default is Never expire. Every log group created without an explicit policy keeps growing indefinitely. Even so, shortening retention usually moves the bill less than teams expect. You pay ingestion once per GB, every month, on all new data. In most accounts the per-GB ingestion price is much higher than the per-GB-month storage price.

Illustrative estimate. These are not real AWS prices. Assume an account ingests 1,000 GB per month and keeps 30 days of retention, so roughly 1,000 GB is stored at any time. Use placeholder prices of 20 units per GB ingested and 1 unit per GB-month stored. Ingestion then costs 20,000 units and storage costs 1,000 units, so storage is under 5% of the combined total. Cutting retention to 15 days saves about 500 units, or roughly 2.4% of that total. Reducing ingestion by 20% saves 4,000 units.

The real ratio depends on current pricing and your retention periods, so check it against your own bill. In most accounts ingestion is the line item to attack first.

How do you find the log groups that cost the most?

Start with ingestion, because it is usually the larger charge. Every log group publishes an IncomingBytes metric in the AWS/Logs namespace, with a LogGroupName dimension. Sum it over a representative window, such as the last 7 or 30 days, and sort the results in descending order. CloudWatch Metrics Insights can do this in one query: SELECT SUM(IncomingBytes) FROM SCHEMA("AWS/Logs", LogGroupName) GROUP BY LogGroupName ORDER BY SUM() DESC LIMIT 20. Use a window long enough to include batch jobs and weekly traffic peaks. Otherwise a single quiet day can hide a noisy group.

Next, look at storage. aws logs describe-log-groups returns storedBytes and retentionInDays for each group. If a group has never had a retention policy set, the retentionInDays field is missing, and the group keeps its data indefinitely. To list those groups together with their stored size, run aws logs describe-log-groups --query 'logGroups[?retentionInDays==null].[logGroupName,storedBytes]' --output table. The CLI follows pagination for you, so the output covers the whole region. Run the command in every region you use, because log groups are regional.

Compare the two lists side by side. The ingestion ranking shows where to cut volume at the source. The storage list shows which groups need a retention policy first. Large groups with no retention are the quickest wins.

The ranking also saves engineering time. Spend in most accounts is concentrated in a small number of groups. Fixing those few first lets you avoid tuning hundreds of low-volume Lambda and test groups whose combined cost is too small to matter. Save the output so you can repeat the audit later and see which changes actually reduced cost.

#!/usr/bin/env bash
# List CloudWatch log groups with no retention policy (data kept forever),
# largest stored size first, across every region you use.
set -euo pipefail

# Log groups are regional, so audit each region explicitly.
REGIONS="us-east-1 eu-west-1"   # placeholder: replace with your regions
OUT="no-retention-$(date +%Y-%m-%d).tsv"

printf 'region\tlogGroupName\tstoredBytes\n' > "$OUT"

for region in $REGIONS; do
  # The CLI paginates automatically. A missing retentionInDays compares
  # equal to null, so this filter keeps only groups with no policy.
  aws logs describe-log-groups \
    --region "$region" \
    --query 'logGroups[?retentionInDays==null].[logGroupName,storedBytes]' \
    --output text |
    awk -v r="$region" -F'\t' 'NF { print r "\t" $1 "\t" $2 }'
done | sort -t$'\t' -k3,3nr >> "$OUT"

# Show the 20 largest. Fix these first; the long tail rarely matters.
head -n 21 "$OUT" | column -t -s$'\t'

# Keep the dated file so you can rerun the audit later and compare results.
echo "Full list saved to $OUT"

A shell script that lists log groups with no retention policy in each region, sorts them by storedBytes, and saves a dated report so you can repeat the audit later.

How can you reduce log ingestion cost at the source?

Retention and log class settings only change what happens after a byte arrives. Ingestion is billed per GB as soon as an event is accepted, so a log line you never emit is the only one that costs nothing. Fix volume at the source before tuning anything else.

  • Raise default log levels in production. Ship INFO or WARN by default, not DEBUG. Debug output is often the largest share of volume and the least read.
  • Drop or sample health checks and routine successes. Load balancer probes and repeated 200 responses on hot endpoints produce the same line thousands of times. Filter them out in the logging layer, or keep a fixed sample of 1 in N. Track counts and latency with metrics instead.
  • Don't log full payloads. Request and response bodies inflate volume and can expose personal data. Log an ID, the size, and the fields you actually search on.
  • Prefer compact structured fields. Short, consistent JSON keys cost less than long prose messages and are easier to query later.

Trade-off: aggressive sampling can delete the evidence you need during an incident. Never sample errors. Keep full stack traces and request context for failures. Make log levels adjustable at runtime, through configuration or a feature flag, so you can turn on verbose logging for one service without a redeploy.

Illustrative estimate (assumed numbers, not a benchmark):

monthly saving = GB ingested per month × fraction removed × ingestion price per GB

Assume a service ingests 500 GB a month and filtering health checks plus debug output removes 40% of it. That saves 200 GB × your regional per-GB price, every month. The engineering cost is usually a filter and a level change in the shared logging library, done once.

import json, logging, os, random, signal

class SourceReductionFilter(logging.Filter):
    """Drop health checks and sample routine successes. Never sample problems."""
    def __init__(self, sample_rate=0.01):
        super().__init__()
        self.sample_rate = sample_rate  # keep roughly 1 in 100 successful requests

    def filter(self, record):
        if record.levelno >= logging.WARNING:
            return True  # warnings and errors are always kept at full fidelity
        path = getattr(record, "path", None)
        if path in ("/health", "/healthz"):
            return False  # load balancer probes: count them with metrics instead
        if path is not None and getattr(record, "status", 0) < 400:
            return random.random() < self.sample_rate
        return True

class CompactJson(logging.Formatter):
    """Short keys, no payload bodies: only IDs, sizes and fields you search on."""
    def format(self, record):
        doc = {"t": self.formatTime(record), "lvl": record.levelname, "msg": record.getMessage()}
        for key in ("req_id", "path", "status", "bytes"):
            if hasattr(record, key):
                doc[key] = getattr(record, key)
        if record.exc_info:
            doc["exc"] = self.formatException(record.exc_info)  # keep full stack traces
        return json.dumps(doc, separators=(",", ":"))

handler = logging.StreamHandler()
handler.addFilter(SourceReductionFilter())
handler.setFormatter(CompactJson())
root = logging.getLogger()
root.addHandler(handler)
root.setLevel(os.environ.get("LOG_LEVEL", "INFO"))  # production default is INFO, not DEBUG

# Toggle DEBUG on a running process (kill -USR1 <pid>) without a redeploy.
signal.signal(signal.SIGUSR1, lambda *_: root.setLevel(
    logging.INFO if root.level == logging.DEBUG else logging.DEBUG))

log = logging.getLogger("api")
log.info("request", extra={"req_id": "r-123", "path": "/orders", "status": 200, "bytes": 512})

A shared Python logging setup that drops health checks, samples successful requests, always keeps warnings and errors with stack traces, emits compact JSON, and allows the log level to be toggled at runtime with SIGUSR1 on Unix.

How should you set a log retention policy on every log group?

A new log group has no retention setting, so its data never expires. Storage then grows with every day of ingestion. Every group should carry an explicit retention value. For a one-off fix, run aws logs put-retention-policy --log-group-name <name> --retention-in-days <n>. The days value must be one of the fixed periods CloudWatch accepts.

The durable fix is to declare each log group in IaC with retention set, before any service writes to it. Lambda and several other services write to predictable names such as /aws/lambda/<function-name>. If that group doesn't exist, the service creates it as a Standard-class group that never expires. Pre-creating the group in IaC gives you control over retention and log class.

If a group was already auto-created, deployment fails with an "already exists" conflict. You can import the existing group into your IaC state, or delete it and let IaC recreate it. Deleting it discards its data.

Run a periodic drift check. List all groups with describe-log-groups and flag any group that has no retentionInDays field.

Tier Example groups Typical retention
Debug / dev Feature branches, sandbox accounts 1–7 days
Application Production service logs 14–90 days
Audit-relevant Auth, access, change events Set by compliance owners

Trade-offs:

  • Retention changes mainly reduce storage cost. They do nothing for ingestion cost.
  • Shortening retention permanently deletes data older than the new period.
  • Before changing retention, owners must confirm legal, security and audit requirements.
  • Deletion is not instantaneous at the retention boundary. Expired events can remain queryable for a short time after they pass it.
# CloudFormation: create the Lambda log group before the function exists,
# so Lambda never auto-creates a Standard-class group that never expires.
Parameters:
  FunctionName:
    Type: String
    Default: orders-api
  ExecutionRoleArn:
    Type: String  # existing IAM role for the function (placeholder)

Resources:
  OrdersApiLogGroup:
    Type: AWS::Logs::LogGroup
    # Retain keeps the group if the stack is deleted. If it is removed by mistake,
    # its data is not deleted with it.
    DeletionPolicy: Retain
    UpdateReplacePolicy: Retain
    Properties:
      # Must match the name Lambda writes to: /aws/lambda/<function-name>
      LogGroupName: !Sub /aws/lambda/${FunctionName}
      RetentionInDays: 30        # application tier; must be an accepted period
      LogGroupClass: STANDARD    # set explicitly instead of relying on the default
      Tags:
        - Key: retention-tier
          Value: application

  OrdersApiFunction:
    Type: AWS::Lambda::Function
    DependsOn: OrdersApiLogGroup  # group exists before the function can log
    Properties:
      FunctionName: !Ref FunctionName
      Runtime: python3.12
      Handler: index.handler
      Role: !Ref ExecutionRoleArn
      Code:
        ZipFile: |
          def handler(event, context):
              print("ok")
              return {"status": 200}

A CloudFormation template that declares the Lambda function's predictable /aws/lambda/ log group with explicit retention and log class, so Lambda does not auto-create a group that never expires.

When should you use the Infrequent Access log class, and when should you avoid it?

The Infrequent Access (IA) log class charges a lower per-GB ingestion price than Standard in exchange for a reduced feature set. It suits logs you must keep but rarely read, such as audit trails, verbose debug output retained for post-incident review, or compliance archives. You can still query IA log groups with Logs Insights, so occasional investigation works without exporting data elsewhere. AWS has changed the feature list since the class launched, so check the current support matrix in the AWS documentation before choosing it.

Capability Standard Infrequent Access
Logs Insights queries Yes Yes
Metric filters Yes No
Subscription filters (streaming to other services) Yes No
Embedded Metric Format extraction Yes No
Retention policies Yes Yes
Intended access pattern Frequent, real-time, alerting Occasional, ad hoc

When to avoid it

Keep a log group on Standard if anything depends on processing logs as they arrive. This includes metric filters that back CloudWatch alarms, subscription filters that stream to a SIEM or a processing pipeline, and applications that publish metrics through Embedded Metric Format. If you move one of these groups to IA, ingestion still succeeds and no error appears, but the derived metrics stop updating and the downstream consumers stop receiving data. Your alarms can then sit in an OK or INSUFFICIENT_DATA state during a real outage.

Querying can also cancel out the savings. Logs Insights charges by data scanned, and that charge applies to IA groups too. A group that dashboards or engineers query many times a day can cost more in scan charges than the ingestion discount saves. Classify each group by its actual query volume, taken from usage data, and not by assumptions about how often it is read.

How do you migrate an existing log group to a different log class?

Log class is fixed when a log group is created. There is no API call or console setting that converts an existing Standard group to Infrequent Access, or back. Log group names are also unique within a Region and account, so the new group needs a new name. Migration means running a parallel group, not converting the old one.

  1. Create the target group with the new class and an explicit retention policy at creation time, so it never starts in the default never-expire state.
  2. Repoint producers. Update the agent configuration, Lambda logging settings, container log drivers or SDK calls that write to the old name. Roll this out deliberately, because a missed producer keeps paying Standard ingestion rates.
  3. Update everything that references the name. IAM policies scope permissions by log group ARN, so writers and readers need the new ARN or they fail. Dashboards, saved Logs Insights queries, alarms on metric filters and runbooks all point at the old group.
  4. Let the old group age out. Leave its retention policy in place and delete the group once its newest events expire.
 before         cutover                     after retention window
 ------         -------                     ----------------------
 app ──► std    app ─┐   std (read-only,    app ──► ia
                     │   expiring)
                     └─► ia                 std  (deleted)

Until the old group expires, data is split across two groups. Incident queries that span the cutover need to target both groups. Plan the migration for a quiet period, and record the cutover timestamp where on-call engineers will find it.

#!/usr/bin/env bash
# Parallel-group migration from Standard to Infrequent Access.
# Log class cannot be changed in place, so the new group needs a new name.
set -euo pipefail

REGION="us-east-1"
OLD_GROUP="/app/orders-api"        # existing Standard group
NEW_GROUP="/app/orders-api-ia"     # new Infrequent Access group
RETENTION_DAYS=30

# 1. Create the target group with the new class. Set retention right away
#    so the group never sits in the default never-expire state.
aws logs create-log-group \
  --region "$REGION" \
  --log-group-name "$NEW_GROUP" \
  --log-group-class INFREQUENT_ACCESS

aws logs put-retention-policy \
  --region "$REGION" \
  --log-group-name "$NEW_GROUP" \
  --retention-in-days "$RETENTION_DAYS"

# Confirm the class and retention before any producer is repointed.
aws logs describe-log-groups \
  --region "$REGION" \
  --log-group-name-prefix "$NEW_GROUP" \
  --query 'logGroups[].[logGroupName,logGroupClass,retentionInDays]' \
  --output table

# 2-3. Repoint producers and update IAM, dashboards, queries and alarms (not shown).
#      After the last producer switches, record the cutover time for on-call.
echo "Cutover for $OLD_GROUP -> $NEW_GROUP at $(date -u +%Y-%m-%dT%H:%M:%SZ)"

# 4. Later: check the newest event still held in the old group. Delete the
#    group only after that event is older than the old group's retention.
aws logs describe-log-streams \
  --region "$REGION" \
  --log-group-name "$OLD_GROUP" \
  --order-by LastEventTime --descending --max-items 1 \
  --query 'logStreams[0].lastEventTimestamp'
# aws logs delete-log-group --region "$REGION" --log-group-name "$OLD_GROUP"

This AWS CLI script creates an Infrequent Access log group with retention set immediately and records the cutover time. It also checks the old Standard group's newest event before that group is deleted.

What are the alternatives for query spend and multi-year retention?

Logs Insights bills by the volume of data scanned, not by the number of results returned. A query over seven days of a busy group costs roughly seven times as much as the same query over one day, even if both return ten rows. To keep scans small, set the narrowest time range that answers the question and select only the log groups that can contain the answer.

Dashboards need the most care. A widget backed by a query re-runs on every refresh. If it points at a large Infrequent Access or long-retention group, it rescans that data each time it refreshes, whether or not anyone is looking at it.

Illustrative estimate (assumed numbers, not a benchmark)

Formula: monthly scan cost = GB scanned per run × runs per day × 30 × price per GB

Input Wide widget Narrowed widget
GB scanned per run 50 2
Runs per day (5-minute refresh) 288 288
Assumed price per GB $0.005 $0.005
Monthly cost $2,160 $86.40

For multi-year retention requirements, compare these two approaches:

Factor Keep in CloudWatch Export to S3 with lifecycle rules
Storage cost for long-lived data Higher Often lower, especially with tiering to colder storage classes
Native Logs Insights queries Yes No; needs a separate query tool
Pipeline to maintain None Export tasks or a streaming pipeline
Data availability Immediate Delayed
Access control Log group IAM policies Separate bucket policies and IAM

After any change, re-check IncomingBytes and storedBytes for each affected group. Check them again after a full billing cycle has passed. Treat every calculation above as an illustrative estimate, not a promised saving.

How OneEnv can help

Disclosure: OneEnv is the product behind this blog.

OneEnv (oneenv.io) is a workspace-based governance platform for the things services share — environment variables, database structure, API routes and validation rules. A change to a shared value is staged on a branch, reviewed once, and then every affected service explicitly approves or rejects it before it goes live, so you can see who a change touches before it ships instead of finding out in production.

Applied to cloudwatch logs cost reduction, this is where it helps:

Saves time

  • Every service that inherits a shared value is listed in one place, so nobody searches repos and chats to find out who is affected.
  • One review per branch replaces coordinating each team separately; each service still keeps its own approve/reject.
  • Changing a shared variable is one edit and one review instead of editing every service's .env by hand.

Saves cost

  • Fewer production incidents and rollbacks caused by a shared value, route or column changing under a consumer unannounced.
  • Less duplicated configuration to maintain: one shared definition inherited by many services.
  • A wrong or stale value is caught at review time by the services it affects, not by a failed deploy.
  • OneEnv has a free plan (1 workspace, unlimited services and members, full governance), so trying it costs nothing.

These are qualitative benefits, not measured results — how much you save depends on how many services share the value.

Want to try it? Start at oneenv.io.