automatic management of aws cloudwatch alarms

[kaykhan]

Describe the feature you are requesting

Whenever a load balancer is created / modified. I would like to automatically create a set of cloudwatch alarms and also have these alarms deleted if the load balancer is destroyed.

Motivation
SOC2 compliance requires that we have a set of default alarms attached to our load balancer / target groups.

ApplicationELB' (Application Elastic Load Balancing) > 'Per AppELB, per AZ Metrics' > 'HTTPCode_ELB_5XX_Count' metric
ApplicationELB > 'Per AppELB, per AZ, per TG Metrics' >  'TargetResponseTime' metric
ApplicationELB' (Application Elastic Load Balancing) > 'Per AppELB, per TG Metrics' > 'UnhealthyHostCount'

Describe the proposed solution you'd like

perhaps some annotation allowing us to set 1 more alarms which cover the above 3 usecases.

service.beta.kubernetes.io/aws-load-balanacer-alarms:

Describe alternatives you've considered

The solution i'm working on is a golang lambda which is being fired whenever there are event bridge / cloud trail events. e.g on ec2 creation/termination create the default alarm. I also now need to do this for our alb's and trying to get my head around how that's going to work

Contribution Intention (Optional)

-[ ] Yes, I am willing to contribute a PR to implement this feature
-[ ] No, I cannot work on a PR at this time

related: aws/karpenter-provider-aws#7824

[shraddhabang]

@kaykhan This is a fair ask and would be a great value add to the project. Thank you for bringing this up. We will try to see if we can take this up in future. Any community contributions are always welcome. :)