Just wanted to share (related to Karpenter):
I've been highly anticipating the
0.28.0 release of Karpenter —the one which has a huge internal refactor on how to handle edge cases for nodes not registering properly. When to terminate unregistered nodes, etc. This release has been taking several months —evidently because of such a huge refactor— and the Karpenter maintainers have cut several release candidates for it. The last time they even had to cut release candidates was back in
0.14.0.
But I
found out that
v0.27.3 has seemingly backported from the
0.28.0 release candidates some very important fixes for handling node lifecycle. For example handling nodes that no longer exist in the cloud provider (i.e. EC2 instances that are already terminated) but the k8s API cannot delete the node object, e.g. pods stuck in terminating because of PDBs,
do-not-evict annotations, etc.
So if you're not already on the latest
0.27.x release, I highly recommend the upgrade.