Support for NLB Subnet Updates

[mhawley1230]

Describe the feature you are requesting

Replace an NLB subnet in an existing AZ without manually removing and replacing

Motivation

AWS recently put out an article on how to NLB subnets can be updated https://aws.amazon.com/blogs/networking-and-content-delivery/exploring-new-subnet-management-capabilities-of-network-load-balancer/

However, an EKS environment uses a large number of IPs and environments may have specific restrictions for account, region, and uptime - especially in production environments.

If AZs are restricted (ex. can only use us-east-1a, 1b, 1c and not 1d, 1e, and 1f) and a subnet change needs to be made (ex. ran out of IP addresses), it makes sense for an existing NLB subnet to be changed without the ability to manually remove an existing subnet. Conflicting with the method from the Subnet Management feature.
Subnet changes within the same Availability Zone must be independent actions. You first complete removing the existing subnet, then you can add the new subnet.

The AWS LB Controller does not support this update order in these versions:

• Helm chart v1.8.1
• Docker image v2.7.1

Results in the following error:

You cannot specify an additional subnet from an Availability Zone that is already associated with the load balancer...

Describe the proposed solution you'd like
Have the load-balancer-controller handle subnet replacement in a single action.

• Remove the subnet from the AZ first.
• Wait for the update to complete.
• Add the new subnet in the same AZ.

[zac-nixon]

Hi. Unfortunately, this behavior is "by-design". The ELB API does not allow this seamless transition for NLB subnets.

[kevarr]

Hi @zac-nixon, is this to say that it's not possible to update the AWS LB Controller to sequence the SetSubnets API calls per the constraints provided in the docs?