Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Cannot use AZWI In AzureUSGovernment #1302

Open
BenjaminWinks opened this issue Mar 26, 2024 · 0 comments
Open

Cannot use AZWI In AzureUSGovernment #1302

BenjaminWinks opened this issue Mar 26, 2024 · 0 comments
Labels
bug Something isn't working

Comments

@BenjaminWinks
Copy link

BenjaminWinks commented Mar 26, 2024

We cannot use azwi to create either a service account or a federated identity in the AzureUSGovernment cloud. The endpoint that is being hit to look up our subscription-id is the public endpoint, not the AzureUSGovernment endpoint and thus when our subscription-id is queried it returns a 404. When using --azure-env to set it to AzureUSGovernment or AzureUSGovernmentCloud, the setting is ignored.

Steps To Reproduce

azwi serviceaccount create phase sa \
  --aad-application-name "${APPLICATION_NAME}" \
  --service-account-namespace "${SERVICE_ACCOUNT_NAMESPACE}" \
  --service-account-name "${SERVICE_ACCOUNT_NAME}" \
  --subscription-id "${SUBSCRIPTION_ID}"

and

azwi serviceaccount create phase federated-identity \
  --aad-application-name "${APPLICATION_NAME}" \
  --service-account-namespace "${SERVICE_ACCOUNT_NAMESPACE}" \
  --service-account-name "${SERVICE_ACCOUNT_NAME}" \
  --service-account-issuer-url "${SERVICE_ACCOUNT_ISSUER}"
  --subscription-id "${SUBSCRIPTION_ID}"

Expected behavior
Based on the documentation found here the following output or some version thereof should be generated, but it doesn't work.

INFO[0000] No subscription provided, using selected subscription from Azure CLI: REDACTED
INFO[0032] [federated-identity] added federated credential  objectID=REDACTED subject="system:serviceaccount:default:workload-identity-sa"

Logs
First is for attempting to create a service account:

Mon, 25 Mar 2024 15:33:24 EDT  cloud/azureclient.go:202  Resolving tenantID  {"subscriptionID": xxxxxxx-xxxx-xxxx-xxxx-xxxxxxxxxxx"}
Mon, 25 Mar 2024 15:33:24 EDT  transport/round_trippers.go:466  curl -v -XGET  -H "Accept: application/json" -H "User-Agent: azsdk-go-armsubscriptions/v1.1.1 (go1.21.7; Windows_NT)" -H "Authorization: Bearer" 
'https://management.azure.com/subscriptions/xxxxxxx-xxxx-xxxx-xxxx-xxxxxxxxxxx?api-version=2021-01-01'
Mon, 25 Mar 2024 15:33:24 EDT  transport/round_trippers.go:495  HTTP Trace: DNS Lookup for management.azure.com resolved to [{4.150.240.10 }]
Mon, 25 Mar 2024 15:33:24 EDT  transport/round_trippers.go:510  HTTP Trace: Dial to tcp:4.150.240.10:443 succeed
Mon, 25 Mar 2024 15:33:25 EDT  transport/round_trippers.go:553  GET https://management.azure.com/subscriptions/xxxxxxx-xxxx-xxxx-xxxx-xxxxxxxxxxx?api-version=2021-01-01 404 Not Found in 411 milliseconds
Mon, 25 Mar 2024 15:33:25 EDT  transport/round_trippers.go:570  HTTP Statistics: DNSLookup 85 ms Dial 48 ms TLSHandshake 216 ms ServerProcessing 60 ms Duration 411 ms
Mon, 25 Mar 2024 15:33:25 EDT  transport/round_trippers.go:577  Response Headers:
Mon, 25 Mar 2024 15:33:25 EDT  transport/round_trippers.go:580      Pragma: no-cache
Mon, 25 Mar 2024 15:33:25 EDT  transport/round_trippers.go:580      X-Ms-Request-Id: 4d1a05d1-1547-4127-8786-f96c7d144251
Mon, 25 Mar 2024 15:33:25 EDT  transport/round_trippers.go:580      Strict-Transport-Security: max-age=31536000; includeSubDomains
Mon, 25 Mar 2024 15:33:25 EDT  transport/round_trippers.go:580      Cache-Control: no-cache
Mon, 25 Mar 2024 15:33:25 EDT  transport/round_trippers.go:580      Expires: -1
Mon, 25 Mar 2024 15:33:25 EDT  transport/round_trippers.go:580      X-Ms-Correlation-Request-Id: 4d1a05d1-1547-4127-8786-f96c7d144251
Mon, 25 Mar 2024 15:33:25 EDT  transport/round_trippers.go:580      X-Content-Type-Options: nosniff
Mon, 25 Mar 2024 15:33:25 EDT  transport/round_trippers.go:580      Content-Type: application/json; charset=utf-8
Mon, 25 Mar 2024 15:33:25 EDT  transport/round_trippers.go:580      X-Ms-Failure-Cause: gateway
Mon, 25 Mar 2024 15:33:25 EDT  transport/round_trippers.go:580      Date: Mon, 25 Mar 2024 19:33:25 GMT
Mon, 25 Mar 2024 15:33:25 EDT  transport/round_trippers.go:580      Content-Length: 129
Mon, 25 Mar 2024 15:33:25 EDT  transport/round_trippers.go:580      X-Ms-Routing-Request-Id: EASTUS:20240325T193325Z:4d1a05d1-1547-4127-8786-f96c7d144251
Mon, 25 Mar 2024 15:33:25 EDT  transport/round_trippers.go:580      X-Cache: CONFIG_NOCACHE
Mon, 25 Mar 2024 15:33:25 EDT  transport/round_trippers.go:580      X-Msedge-Ref: Ref A: 97520412180E4B30AC98BCA191A30161 Ref B: MNZ221060618031 Ref C: 2024-03-25T19:33:25Z
Error: header "WWW-Authenticate" not found in get subscription response

The next is attempting to create a federated identity:

Tue, 26 Mar 2024 10:42:34 EDT  cloud/azureclient.go:202  Resolving tenantID  {"subscriptionID": "xxxxxxx-xxxx-xxxx-xxxx-xxxxxxxxxxx"}
Tue, 26 Mar 2024 10:42:34 EDT  transport/round_trippers.go:466  curl -v -XGET  -H "Accept: application/json" -H "User-Agent: azsdk-go-armsubscriptions/v1.1.1 (go1.21.7; Windows_NT)" -H "Authorization: Bearer" 
'https://management.azure.com/subscriptions/xxxxxxx-xxxx-xxxx-xxxx-xxxxxxxxxxx?api-version=2021-01-01'
Tue, 26 Mar 2024 10:42:34 EDT  transport/round_trippers.go:495  HTTP Trace: DNS Lookup for management.azure.com resolved to [{4.150.240.10 }]
Tue, 26 Mar 2024 10:42:34 EDT  transport/round_trippers.go:510  HTTP Trace: Dial to tcp:4.150.240.10:443 succeed
Tue, 26 Mar 2024 10:42:34 EDT  transport/round_trippers.go:553  GET https://management.azure.com/subscriptions/xxxxxxx-xxxx-xxxx-xxxx-xxxxxxxxxxx?api-version=2021-01-01 404 Not Found in 347 milliseconds
Tue, 26 Mar 2024 10:42:34 EDT  transport/round_trippers.go:570  HTTP Statistics: DNSLookup 31 ms Dial 46 ms TLSHandshake 197 ms ServerProcessing 70 ms Duration 347 ms
Tue, 26 Mar 2024 10:42:34 EDT  transport/round_trippers.go:577  Response Headers:
Tue, 26 Mar 2024 10:42:34 EDT  transport/round_trippers.go:580      X-Ms-Failure-Cause: gateway
Tue, 26 Mar 2024 10:42:34 EDT  transport/round_trippers.go:580      X-Ms-Correlation-Request-Id: 2894f6bf-6f8a-4d28-998d-8e7c0ec14e8b
Tue, 26 Mar 2024 10:42:34 EDT  transport/round_trippers.go:580      X-Content-Type-Options: nosniff
Tue, 26 Mar 2024 10:42:34 EDT  transport/round_trippers.go:580      Cache-Control: no-cache
Tue, 26 Mar 2024 10:42:34 EDT  transport/round_trippers.go:580      X-Ms-Request-Id: 2894f6bf-6f8a-4d28-998d-8e7c0ec14e8b
Tue, 26 Mar 2024 10:42:34 EDT  transport/round_trippers.go:580      Date: Tue, 26 Mar 2024 14:42:33 GMT
Tue, 26 Mar 2024 10:42:34 EDT  transport/round_trippers.go:580      Pragma: no-cache
Tue, 26 Mar 2024 10:42:34 EDT  transport/round_trippers.go:580      Expires: -1
Tue, 26 Mar 2024 10:42:34 EDT  transport/round_trippers.go:580      X-Ms-Routing-Request-Id: EASTUS2:20240326T144234Z:2894f6bf-6f8a-4d28-998d-8e7c0ec14e8b
Tue, 26 Mar 2024 10:42:34 EDT  transport/round_trippers.go:580      Strict-Transport-Security: max-age=31536000; includeSubDomains
Tue, 26 Mar 2024 10:42:34 EDT  transport/round_trippers.go:580      X-Msedge-Ref: Ref A: 74A19B4E725345C18A15E8879D842679 Ref B: MNZ221060609025 Ref C: 2024-03-26T14:42:34Z
Tue, 26 Mar 2024 10:42:34 EDT  transport/round_trippers.go:580      Content-Type: application/json; charset=utf-8
Tue, 26 Mar 2024 10:42:34 EDT  transport/round_trippers.go:580      X-Cache: CONFIG_NOCACHE
Tue, 26 Mar 2024 10:42:34 EDT  transport/round_trippers.go:580      Content-Length: 129
Error: header "WWW-Authenticate" not found in get subscription response

Environment
Azure USGoverment GCC High IL-4

  • Kubernetes version (use kubectl version):
Client Version: v1.28.4
Kustomize Version: v5.0.4-0.20230601165947-6ce0bf390ce3
Server Version: v1.26.10
  • Cloud provider or hardware configuration: AzureGov using AKS
  • OS (e.g: cat /etc/os-release):
PRETTY_NAME="Ubuntu 23.04"
NAME="Ubuntu"
VERSION_ID="23.04"
VERSION="23.04 (Lunar Lobster)"
VERSION_CODENAME=lunar

NOTE: This also doesn't work with Windows 10 using azwi for powershell either.

  • Kernel (e.g. uname -a): Linux <HOSTNAME> 5.4.72-microsoft-standard-WSL2 #1 SMP Wed Oct 28 23:40:43 UTC 2020 x86_64 x86_64 x86_64 GNU/Linux
  • Install tools: I installed this by grabbing the release from github: wget https://github.com/Azure/azure-workload-identity/releases/download/v1.2.1/azwi-v1.2.1-linux-amd64.tar.gz and then moving to /usr/bin/azwi

Additional context
When attempting to run the commands on the guide we can tell that it's reaching out to the wrong endpoint, hence the 404 when looking up our subscription-id. We are attempting to set this up in order to support the usage of external secrets operator by following their instructions to use azwi. I noticed there's a ticket open already about it not working in non-public clouds (#566 ) We are also not sure why Error: header "WWW-Authenticate" not found in get subscription response is being returned in the logs, but since it doesn't match the anticipated outcome in the guide, we presume it's due to the cloud API endpoint being incorrect.

@BenjaminWinks BenjaminWinks added the bug Something isn't working label Mar 26, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
bug Something isn't working
Projects
None yet
Development

No branches or pull requests

1 participant