Spot Node Pools: Cheap Compute With a Catch

Spot node pools are the closest thing AKS has to free money: Azure's spare capacity at a steep discount. The catch is right there in the deal: Azure can reclaim a spot node at any time, with about 30 seconds' notice. That single sentence decides everything about how you use them.

Good on spot:

• Stateless services that already run multiple replicas and reschedule cleanly.
• Batch jobs, CI builds, and data processing that can restart without anyone caring.
• Dev and test workloads where a brief blip is a non-event.

Wrong on spot:

• Stateful workloads holding data on the node, where eviction means loss or a painful recovery.
• Latency-critical, single-replica, or "must always be up" services.
• Anything where a 30-second eviction turns into a customer-visible incident.

The mistake isn't using spot. It's letting the wrong pods land on it. The discipline is two parts:

1. Taint the spot pool so nothing schedules there unless it explicitly tolerates eviction. Opt-in, never default. A pod ends up on spot because its author decided it could, not because the scheduler ran out of room.
2. Handle eviction like you mean it: multiple replicas, PodDisruptionBudgets, and graceful SIGTERM handling so a reclaimed node is a shrug, not a page.

Done right, spot becomes a tier in your node-pool architecture: the interruptible top of the stack, soaking up the work that genuinely doesn't mind being interrupted, while your steady baseline runs on reserved or on-demand capacity below it.

Spot isn't a hack to bolt on for a quick saving. It's a deliberate place in your cluster for workloads that are honest about being interruptible. Treat it that way and the discount is real and free. Treat it as "cheaper nodes for everything" and you've bought yourself a discount on outages.