| # | Category | Cluster | Owner | Monthly saving | Annual saving | Confidence |
|---|---|---|---|---|---|---|
| 1 | non spot job cluster | job-feature-pipeline | ml-svc@acme.com | $2,396/mo | $28,754/yr | 82% |
|
Evidence: Job cluster 'job-feature-pipeline' runs on on-demand VMs ($3,686 VM cost in 30 days). Batch/ETL jobs can tolerate spot interruption via Databricks' automatic retry. Spot VMs cost 60-80% less. Action: Enable Azure Spot or AWS Spot in cluster configuration (azure_attributes.spot_bid_max_price or aws_attributes.availability = SPOT). Databricks automatically handles spot interruptions with retries for Jobs. |
||||||
| 2 | missing autoscaling | analytics-prod | wei@acme.com | $1,924/mo | $23,093/yr | 78% |
|
Evidence: 'analytics-prod' has 8 fixed workers with no autoscaling. Fixed-size clusters overprovision during off-peak hours. Estimated 35% waste from idle workers during low-utilisation windows. Action: Enable autoscaling in cluster configuration. Set min_workers = 1-2 (enough to keep the cluster warm) and max_workers = current fixed count. Also enable auto-termination (30 min) for interactive clusters. |
||||||
| 3 | idle interactive cluster | [ds-team] exploration | priya@acme.com | $698/mo | $8,381/yr | 95% |
|
Evidence: Cluster '[ds-team] exploration' has been running for 48h with no query or job activity. Blended cost (DBU + VM): $698/month. Action: Terminate the cluster. If needed for interactive work, enable auto-termination (15-30 min) so it stops automatically when idle. |
||||||
| 4 | orphaned cluster | tmp-test-cluster-DO-NOT-DELETE | intern@acme.com | $577/mo | $6,921/yr | 88% |
|
Evidence: 'tmp-test-cluster-DO-NOT-DELETE' has been running for 30 days with no activity in the last 7+ days. Owner: intern@acme.com. Blended cost: $577 over 30 days. Action: Contact the owner and confirm whether the cluster is still needed. If not, terminate immediately. Set a cluster policy that enforces auto-termination on all interactive clusters. |
||||||
| 5 | non photon sql workload | bi-dashboards | bi@acme.com | $462/mo | $5,543/yr | 65% |
|
Evidence: 'bi-dashboards' ($1,540/30d) runs without Photon. If this cluster runs SQL queries, aggregations, or Delta MERGE operations, Photon acceleration reduces runtime by 2-4×, lowering cluster-hours and therefore both DBU and VM cost. Action: Enable Photon in the cluster runtime configuration. Note: Photon consumes DBUs at a higher rate but reduces cluster time by 2-4× for SQL — net saving for SQL-heavy workloads. Do NOT enable for ML training or Python UDF-heavy workloads. |
||||||
| 6 | all purpose as job cluster | nightly-etl-attach | raj@acme.com | $223/mo | $2,673/yr | 91% |
|
Evidence: 95% of 'nightly-etl-attach' runtime comes from the Jobs scheduler, but it runs on All-Purpose Compute ($0.55/DBU on Premium). Jobs Compute costs $0.30/DBU. Action: Convert the Databricks Job to use a 'New job cluster' with a Jobs Compute policy instead of attaching to an existing All-Purpose cluster. Terminate the All-Purpose cluster once migrated. |
||||||