AWS Glue Flex Execution

TL;DR: Flex Execution (late 2022) runs the exact same Glue job on lower-priority capacity for ~34% less per DPU-hour. The only thing you give up is a guaranteed-instant start — Flex jobs queue behind standard jobs and typically begin within a few minutes. For the batch-ETL majority of Glue spend (overnight, weekly, backfill, reconciliation), where the output is consumed hours later, that scheduler precision is a phantom requirement and Flex is effectively free money.

The numbers

• Standard ~$0.44/DPU-hour → Flex ~$0.29/DPU-hour (us-east-1) — the ~34% cut. A DPU is a bundle of CPU + memory + I/O, billed per second.
• Observed start delay: seconds to a few minutes typically; 3–10 min common; occasionally 10–20 min at AWS peak; rarely longer during region-wide crunches.
• Worked example: 10 DPUs × 2 hrs/day = 600 DPU-hours/mo → standard $264/mo vs Flex $174/mo ($90/mo, ~$1,080/yr on one job). A team with $5,000/mo of Glue spend converting half its jobs saves ≈ $5,000 × 50% × 34% = $850/mo.
• Field examples: nightly fintech log processing saved ~$1,200/mo (never missed the 6 AM dashboard); a weekly warehouse rebuild saved >$3,000/mo; monthly HIPAA reports on a 48-hour window saved ~$800–900/run.

Do this

1. List Glue jobs sorted by monthly DPU-hours and flip the top spenders first — the 34% scales linearly with what each job costs today.
2. For each, ask when the output is actually consumed — if there's >30 min of slack between "could start" and "must finish," it's a Flex candidate.
3. Turn it on — one setting, no code change: Job details → Advanced properties → Execution class → Flex, or CLI --execution-class FLEX.
4. Test on one job for two weeks — graph submitted-vs-started timestamps (submission time in CloudWatch Logs, actual start in the job run record); most teams find the delay smaller than feared.
5. Alarm on the delta if a sustained queue wait would ever break a downstream SLA, then flip the rest of the easy wins.

Gotchas

• Only the start is delayed — once a Flex job begins it's identical to a standard job: same engine, same retries, same failure modes. There's no lower-tier "Flex hardware."
• The deadline test is "when must it finish," not "when must it start" — most "urgent" jobs are merely scheduled at a habitual time.
• Bursty peak-hour delays (10–20 min) are the realistic worst case for eligible jobs — size your slack against the 95th-percentile wait, not the median.

Skip this if

• The pipeline is real-time or near-real-time (fraud detection, streaming joins, live dashboards), sits in a tight Step Functions chain with per-step SLAs, or feeds freshness-critical/user-facing data.
• You want to compound the savings on the same jobs — pair with Glue Auto Scaling (right-size DPU count) and AWS Glue Job Bookmarks (skip already-processed data on incremental runs); the three stack roughly additively.