TL;DR: CloudWatch has no account-wide retention default — every new log group is created as Never Expire, and there's no console setting to change that. So five-year-old accounts carry hundreds of groups quietly billing $0.03/GB-month forever, including logs for Lambdas deleted in 2023. Setting retention deletes old events, never the group or its filters — zero application risk, savings within the week.
The numbers
- Ingestion ~$0.50/GB (the big line, not fixed by retention) · Storage ~$0.03/GB-month, compounding forever
- One 10 GB/month Lambda kept 5 years = 600 GB ≈ $18/month — for one function
- Field examples: a startup found half its $200/month storage was three Lambdas decommissioned 18 months prior → $34/month after a two-hour cleanup; a SaaS with 4 TB of VPC Flow Logs at $900/month went to ~$80/month with 7-day CloudWatch retention + S3/Glacier for the audit year
- The honest heuristic: almost nobody reads logs older than 30 days — past that they exist for audit, forensics, or by accident
Do this
-
Find the offenders: CloudWatch Logs console sorted by storage, or:
aws logs describe-log-groups \ --query "logGroups[?!retentionInDays].[logGroupName,storedBytes]" --output table -
Set 30 days on the top offenders now, then bulk-fix the rest of the region:
aws logs describe-log-groups --query "logGroups[?!retentionInDays].logGroupName" --output text \ | xargs -n1 -I{} aws logs put-retention-policy --log-group-name {} --retention-in-days 30The script can't delete a group — only set a policy. Old events start aging out within a day.
-
Delete orphaned groups outright (services long gone) — faster than waiting for retention.
-
Route long-tail retention to S3, not CloudWatch: Flow Logs and CloudTrail data events get 7–90 days in CloudWatch + a subscription filter to S3/Glacier for the compliance year. CloudWatch is for active observability windows; S3 is for archives.
-
Enforce at creation:
retention_in_daysin every Terraform/CDK log-group module; a ~30-line Lambda onCreateLogGroupevents as the safety net (Lambda auto-createsNever Expiregroups on first invocation if IaC didn't pre-create them); AWS Config rulecw-loggroup-retention-period-checkfor reporting.
Gotchas
- Retention doesn't touch ingestion. If the bill is mostly the $0.50/GB collection charge, the fix is filtering/sampling at the source — a separate exercise.
- Lambda's auto-created log groups are the classic leak — pre-declare them in IaC with retention set.
- Retention is per group, not per stream; different durations need different groups.
- Suggested defaults by type: app/debug logs 30 days; API access logs 30–90; Flow Logs/CloudTrail data events 90 in CloudWatch then S3; CloudTrail management events 7 (Event History + the S3 trail already cover you).
Skip this if
- You genuinely run Logs Insights queries against months-old data (Insights doesn't work on S3) — keep those specific groups long.
- A regulator explicitly requires queryability from CloudWatch (rare).
- The group is tiny — a 1 GB group isn't worth ceremony, though the bulk script fixes it for free anyway. For the 30-day-to-1-year keep-but-rarely-query zone, see CloudWatch Logs Infrequent Access.