Support AWS EKS IRSA (AWS_WEB_IDENTITY_TOKEN_FILE and AWS_ROLE_ARN)

[paulopontesm]

Problem

Consul uses this library for Cloud Auto-join. On AWS EKS, if we follow AWS best practices and "Restrict access to the instance profile assigned to the worker node" by configuring our nodes with http_put_response_hop_limit=1, consul won't be able to get credentials from AWS EKS IRSA.

In a nutshell, this is how I think it works:

1. When http_put_response_hop_limit=2 if we do http://169.254.169.254/latest/meta-data/
   • Inside a pod that has a K8s ServiceAccount WITH the correct eks.amazonaws.com/role-arn annotation -> The workflows should get credentials from the variables AWS_WEB_IDENTITY_TOKEN_FILE and AWS_ROLE_ARN. We want this 👍
   • Inside a pod with a K8s ServiceAccount WITHOUT an annotation -> We see the metadata from the ⚠️EC2 instance⚠️ . Ideally, we don't want this, but this is what is supported by Consul.
2. When http_put_response_hop_limit=1 if we do http://169.254.169.254/latest/meta-data/
   • Inside a pod that has a K8s ServiceAccount WITH the correct eks.amazonaws.com/role-arn annotation -> The workflows should get credentials from the variables AWS_WEB_IDENTITY_TOKEN_FILE and AWS_ROLE_ARN. We want this 👍
   • Inside a pod that has a K8s ServiceAccount WITHOUT an annotation -> Not available 🤷

Related issue: hashicorp/consul#8532