IoT Core Message Filtering

TL;DR: IoT Core ingestion is trivially cheap (~$1/M messages), but every message that fires a Lambda, a DynamoDB write, a Kinesis put, or an S3 PUT pays for that service too — a $4 IoT bill routinely balloons past $200/mo once you count the fan-out. The Rules Engine (built in since 2015) lets you write a SQL-like WHERE clause so only messages you care about get forwarded; everything else is evaluated and dropped at zero downstream cost. If 80% of your traffic is noise, that one clause is a 50–90% cut.

The numbers

• IoT Core: ~$1 per million messages (us-east-1). 100 devices at 1 msg/min ≈ 4.3M messages/month — IoT volume gets enormous fast.
• The bill is downstream: filter early and the saving compounds across every consumer (Lambda + DynamoDB + Kinesis + S3).
• Worked example — 500 warehouse sensors, temp/humidity/motion every 30s = 1.44M msgs/day. Filter to anomalies only and ~50K/day get through: ~96% fewer downstream invocations and writes.
• Field examples: a 2,000-sensor campus (11.5M msgs/day) filtered to anomalies dropped Lambda 95% and saved $200–$300/mo on one rule; a 200-van GPS fleet went **$400 → ~$120/mo** with speed > 0 plus a bounding box.

Do this

1. Write the filter as a SQL WHERE against the JSON body — only matching messages forward, the rest vanish:

   SELECT * FROM 'warehouse/sensors/+'
   WHERE temperature > 80 OR temperature < 50 OR humidity > 70 OR motion = true
   

2. Geo-filter location streams so you don't process out-of-footprint pings: WHERE (lat BETWEEN 40.5 AND 41.5) AND (lon BETWEEN -74.5 AND -73.5).
3. Route by class in one rule — high-priority alerts → Lambda, low-priority logs → cheap S3, noise → dropped.
4. Rate-limit by change, not clock: pair the Rules Engine with Device Shadows to forward only when a reading moves past a threshold (turns per-second chatter into a few hundred msgs/day).
5. Manage rules in CloudFormation/Terraform once past ~5 rules — diff-able changes, rollback, dev/staging/prod consistency.

Gotchas

• Filtered-out messages are gone — not stored, not billed. Usually fine, but if you might retrain models or reconstruct baselines later, sample ~1% of drops to cheap S3 so you keep 99% of the savings and a statistical record.
• Don't leave rejection logging on — CloudWatch Logs on dropped messages costs ingestion fees; enable only while troubleshooting, then turn it off.
• Over-filtering loses the baseline — if you forward only anomalies, "what does normal look like over time" is unrecoverable.
• The Rules Engine console is bare (no autocomplete, awkward testing) — a reason to move to IaC early.

Skip this if

• You're under ~1M messages/month — filtering won't move absolute dollars yet (add it anyway as discipline so the bill stays flat as the fleet 10×'s).
• You're genuinely unsure what's irrelevant — start minimal (dedupe, heartbeats, debug topics), archive the rest to S3, and tighten in a second pass once you know what drives decisions.
• Your real lever is the post-filter stream volume — once noise is gone, right-size Kinesis shards against the real throughput.