Node Pools Are an Architecture, Not a Setting

There's a default AKS topology that almost everyone starts with and far too many keep: one node pool, general-purpose VMs, every workload scheduled wherever it lands. It feels like the simple choice. It's actually a pile of deferred decisions wearing the costume of simplicity, and they come due at the worst time.

Node pools are the layer where you tell the cluster what your workloads are. Use them well and the scheduler does your capacity planning, your cost control, and your blast-radius management for you. Ignore them and you do all three by hand, badly, during incidents.

What one flat pool actually costs you

• System and apps share fate. AKS's own critical components (CoreDNS, metrics, the bits that keep the cluster a cluster) run alongside your workloads. A runaway app that starves the node can take down the machinery the whole cluster depends on.
• You pay premium prices for cheap work. Batch jobs, dev workloads, and anything interruptible run on full-price on-demand VMs because there's nowhere cheaper to send them.
• One SKU fits nothing well. A general-purpose size over-serves CPU-bound services and under-serves memory-hungry ones. You round up everywhere and pay for the rounding.

Pools as deliberate architecture

Think of each pool as a contract about a class of workload.

• A dedicated system pool. Small, stable, on-demand, tainted so only system components land there. The cluster's nervous system gets its own quiet room.
• General workload pools for your standard services: right-sized SKUs, autoscaled, the bread and butter.
• A spot pool for interruptible work. Batch, CI, dev, anything that can tolerate eviction. Steered there with taints and tolerations so nothing critical wanders in.
• Specialized pools where the workload demands it: memory-optimized for the cache-heavy service, GPU for the inference job, a compute-optimized pool for the CPU-bound one. The workload's shape picks the silicon.

A node pool is a sentence about a workload: this kind of thing belongs on this kind of machine, bought this way, isolated from that. One flat pool says nothing, so you end up saying it manually, later.

How workloads find the right pool

Two primitives do almost all the work:

• Taints and tolerations keep workloads off pools they don't belong on. The system pool's taint repels apps; the spot pool's taint repels anything that can't survive eviction. Only pods that explicitly tolerate a pool can schedule there.
• Labels and node selectors / affinity pull workloads toward the right pool: the GPU job to the GPU nodes, the memory-bound service to the memory-optimized SKU.

Together they turn "where does this run" from a scheduling accident into a declared intent.

The takeaway

The flat single-pool cluster isn't simpler. It's just undecided, and the cluster makes the decisions for you at random. Designing your node pools is how you encode what you actually know about your workloads: what's critical, what's interruptible, what's hungry for memory, what can run cheap. Do it on purpose and your scheduler quietly enforces resilience and cost discipline every minute of every day. That's not a setting. That's architecture.