-
Notifications
You must be signed in to change notification settings - Fork 4
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
[BUG] VaultException: Configured token cannot be verified. It is most likely expired or invalid. #67
Comments
To verify the token, the client issues a lookup request to the server and checks the status code of the response. If it is not 200, it interprets this as the token being invalid. It would be very interesting to see what the response is exactly. Unfortunately, there is no logging at that line. Can you add it and report back what it says? You should find the relevant file at diff --git a/src/saltext/vault/utils/vault/factory.py b/src/saltext/vault/utils/vault/factory.py
index 9978efb..ccad692 100644
--- a/src/saltext/vault/utils/vault/factory.py
+++ b/src/saltext/vault/utils/vault/factory.py
@@ -621,6 +621,7 @@ def _fetch_token(config, opts, token_cache, unwrap_client, force_local=False, em
# lookup and verify raw token
token_info = unwrap_client.token_lookup(embedded_token, raw=True)
if token_info.status_code != 200:
+ log.error(f"Token lookup failed! status: {token_info.status_code} text: {token_info.text}")
raise VaultException(
"Configured token cannot be verified. It is most likely expired or invalid."
) You might need to restart the daemon for the change to take effect. |
@ichilton Since I did not hear back from you, did you find the cause? If not, you can try the new release, which at least logs the API response to aid in debugging. |
Hi @lkubb, Sorry, I completely missed these replies! I just came back to this in the hope that the problem might have gone away / been fixed, but seeing the same. I upgraded to v1.1.1 as you suggested and I see the same error, but with an indication that it's getting a 503 response.
Any ideas? Thanks, Ian |
Hi, Ok, tracked down the 503 error - i've got nginx in front of vault, for SSL termination and as a proxy. I had this:
Vault is only listening on v4:
So I was getting this in the nginx error.log:
So I fixed it with:
Now I get a 403 error...
Any ideas? - or is there a way of seeing why vault would be doing that? If I do Thanks, Ian |
Hi @lkubb, If I do a request to that URL from the salt-master, it does succeed. VAULT_TOKEN environment var is set with the value from:
You're not doing a HTTP HEAD request are you?
Thanks, Ian |
tcpdump'ing port 8200 on the vault server, I can see the raw HTTP requests hitting vault. I can see that the token used in the token-lookup request is not the token from the config file (interestingly it does have the same first few characters - but then is otherwise different). I was then thinking it must be the token for the minion that vault issues by calling However, there is no sign of any such call to vault - the only call is the So where is it getting the token from?? Thanks, Ian |
No worries!
Awesome :)
Permission denied on this endpoint means the token the minion uses is invalid for that server.
The token lookup is only done when using local configuration, not when a minion receives a freshly issued token from the master. I would guess this is where the issue can be found: Since you're using Edit:
|
Soooo, I just found some old vault config hidden in a file under /etc/salt/master.d, which explains the incorrect token! but now I get this:
Config is now:
Now where am I going wrong? :) Thanks, Ian |
https://salt-extensions.github.io/saltext-vault/topics/basic_configuration.html#prerequisites When using token issuance, you need to ensure your master is allowed to issue
|
Thank you - will read up and try again later! |
@ichilton In case it's still relevant, I added a more detailed description of Token Roles and their purpose to the docs: https://salt-extensions.github.io/saltext-vault/topics/basic_configuration.html#credential-issuance Since the cause of the original issue turned out to be external and the new logging helped the diagnosis, I'm going to close this. Feel free to ask further questions in here or in a discussion. :) Thanks! |
Description
Had a working Salt / Vault setup using the old integration.
Started to get deprecation warnings so have been attempting to switch to the new vault extension.
Have been trying for hours and I can't get past:
Error running 'vault.read_secret': Failed to read secret! VaultException: Configured token cannot be verified. It is most likely expired or invalid.
I get this if I try issuing from the master or supply locally on the client.
The token is definitely valid (tested it direct on vault and tried different ones). Even tried the root token.
If I change the URL, it fails with a different error, so definitely seems to be attempting to connect to valult.
Setup
(Please provide relevant configs and/or SLS files (be sure to remove sensitive info. There is no general set-up of Salt.)
/etc/salt/master.d/vault.conf:
Please be as specific as possible and give set-up details.
Steps to Reproduce the behavior
Expected behavior
Should return the secret.
Versions Report
salt --versions-report
(Provided by running salt --versions-report. Please also mention any differences in master/minion versions.)The text was updated successfully, but these errors were encountered: