(E) 400: Invalid Item UUID

[havard024]

Your environment

Operator Version: 1.5.0

Connect Server Version: 1.5.6

Kubernetes Version: 1.22

What happened?

The connect-api is constantly logging the following error message

{"log_message":"(E) 400: Invalid Item UUID","timestamp":"2022-09-26T05:02:29.537013573Z","level":1,"scope":{"request_id":"fd7339af-ca10-4562-8915-df990ad796d2","jti":"r33ngkrcmnw5y6bzt64qty5qlm"}}

The error message itself does not contain enough information to figure out which uuid / item has an invalid UUID.
I've gone through the 8 OnePasswordItems in our cluster and could not find any errors on them either

What did you expect to happen?

That the log message contains the UUID of the item

[havard024]

I've had a second look at our logs and it looks like the error is logged every time the connect-api container fetches a secret by name from 1password.

The secret is fetched successfully, but there's always an error message like this 400: Invalid Item UUID

{"log_message":"(I) GET /v1/vaults/ty324ecoq7a7pjg6mzb5nir2ge/items?filter=title+eq+%22secret1%22","timestamp":"2022-09-30T08:47:45.823551728Z","level":3,"scope":{"request_id":"e7f4bbc1-dabc-4add-8997-9a4053edc275"}}
{"log_message":"(I) GET /v1/vaults/ty324ecoq7a7pjg6mzb5nir2ge/items?filter=title+eq+%22secret1%22 completed (200: OK)","timestamp":"2022-09-30T08:47:45.829565898Z","level":3,"scope":{"request_id":"1334fc55-216e-9194-93af-177267fb40ce","jti":"r33ngkrcmnw5y6bzt64qty5qlm"}}
{"log_message":"(E) 400: Invalid Item UUID","timestamp":"2022-09-30T08:47:45.829648224Z","level":1,"scope":{"request_id":"ffbf6136-0039-9c84-8e2e-3918761b522d","jti":"r33ngkrcmnw5y6bzt64qty5qlm"}}
{"log_message":"(I) GET /v1/vaults/ty324ecoq7a7pjg6mzb5nir2ge/items/kwwusqdt5dkq62wnzuxhgk37fq","timestamp":"2022-09-30T08:47:45.833174723Z","level":3,"scope":{"request_id":"abc52e84-7f47-4e12-a020-af3d498e4d34"}}
{"log_message":"(I) GET /v1/vaults/ty324ecoq7a7pjg6mzb5nir2ge/items/kwwusqdt5dkq62wnzuxhgk37fq completed (200: OK)","timestamp":"2022-09-30T08:47:45.838043949Z","level":3,"scope":{"request_id":"26e99cf8-0d09-9270-a0c8-8f6f4d97e1ba","jti":"r33ngkrcmnw5y6bzt64qty5qlm"}}

{"log_message":"(I) GET /v1/vaults/ty324ecoq7a7pjg6mzb5nir2ge/items?filter=title+eq+%22secret2%22","timestamp":"2022-09-30T08:47:45.843412822Z","level":3,"scope":{"request_id":"627bac5f-da9b-404c-b627-c362562a8bd1"}}
{"log_message":"(I) GET /v1/vaults/ty324ecoq7a7pjg6mzb5nir2ge/items?filter=title+eq+%22secret2%22 completed (200: OK)","timestamp":"2022-09-30T08:47:45.849813447Z","level":3,"scope":{"request_id":"b69e64d3-d1c9-90bd-b277-ddd724ccbd3a","jti":"r33ngkrcmnw5y6bzt64qty5qlm"}}
{"log_message":"(E) 400: Invalid Item UUID","timestamp":"2022-09-30T08:47:45.850463637Z","level":1,"scope":{"request_id":"ffbf6136-0039-9c84-8e2e-3918761b522d","jti":"r33ngkrcmnw5y6bzt64qty5qlm"}}
{"log_message":"(I) GET /v1/vaults/ty324ecoq7a7pjg6mzb5nir2ge/items/nsgk5zk5gvemhecvbwwtpo3t7y","timestamp":"2022-09-30T08:47:45.852916261Z","level":3,"scope":{"request_id":"7f9d0e41-a473-4f08-b407-f016ad250db5"}}
{"log_message":"(I) GET /v1/vaults/ty324ecoq7a7pjg6mzb5nir2ge/items/nsgk5zk5gvemhecvbwwtpo3t7y completed (200: OK)","timestamp":"2022-09-30T08:47:45.857358561Z","level":3,"scope":{"request_id":"5f210657-cd9f-92de-b2a2-86da6dc0b4a4","jti":"r33ngkrcmnw5y6bzt64qty5qlm"}}

{"log_message":"(I) GET /v1/vaults/ty324ecoq7a7pjg6mzb5nir2ge/items?filter=title+eq+%22secret3%22","timestamp":"2022-09-30T08:47:45.860029115Z","level":3,"scope":{"request_id":"20b694a2-6947-49d3-bda6-7fe87d59edd4"}}
{"log_message":"(I) GET /v1/vaults/ty324ecoq7a7pjg6mzb5nir2ge/items?filter=title+eq+%22secret3%22 completed (200: OK)","timestamp":"2022-09-30T08:47:45.865239409Z","level":3,"scope":{"request_id":"6b42c5e3-83f9-9986-8f3f-6d6317785bca","jti":"r33ngkrcmnw5y6bzt64qty5qlm"}}
{"log_message":"(E) 400: Invalid Item UUID","timestamp":"2022-09-30T08:47:45.866262061Z","level":1,"scope":{"request_id":"ffbf6136-0039-9c84-8e2e-3918761b522d","jti":"r33ngkrcmnw5y6bzt64qty5qlm"}}
{"log_message":"(I) GET /v1/vaults/ty324ecoq7a7pjg6mzb5nir2ge/items/7pezesnwzqfcvj43dl73bzrkau","timestamp":"2022-09-30T08:47:45.867712113Z","level":3,"scope":{"request_id":"5c9ffac5-becb-4541-858b-ced05d79e44b"}}
{"log_message":"(I) GET /v1/vaults/ty324ecoq7a7pjg6mzb5nir2ge/items/7pezesnwzqfcvj43dl73bzrkau completed (200: OK)","timestamp":"2022-09-30T08:47:45.875091982Z","level":3,"scope":{"request_id":"3e07c4f8-0e0b-95b5-b385-8df0467e1b1b","jti":"r33ngkrcmnw5y6bzt64qty5qlm"}}

{"log_message":"(I) GET /v1/vaults/ty324ecoq7a7pjg6mzb5nir2ge/items?filter=title+eq+%22secret4%22","timestamp":"2022-09-30T08:47:45.87754808Z","level":3,"scope":{"request_id":"50e41ab3-1a46-4019-9dcb-a486c005f458"}}
{"log_message":"(I) GET /v1/vaults/ty324ecoq7a7pjg6mzb5nir2ge/items?filter=title+eq+%22secret4%22 completed (200: OK)","timestamp":"2022-09-30T08:47:45.882394917Z","level":3,"scope":{"request_id":"1029d577-7db5-9e48-833d-722d75c1d1a4","jti":"r33ngkrcmnw5y6bzt64qty5qlm"}}
{"log_message":"(E) 400: Invalid Item UUID","timestamp":"2022-09-30T08:47:45.882520013Z","level":1,"scope":{"request_id":"ffbf6136-0039-9c84-8e2e-3918761b522d","jti":"r33ngkrcmnw5y6bzt64qty5qlm"}}
{"log_message":"(I) GET /v1/vaults/ty324ecoq7a7pjg6mzb5nir2ge/items/nzirrhwyemdfsrr4ebeghbohai","timestamp":"2022-09-30T08:47:45.885266662Z","level":3,"scope":{"request_id":"825191dc-a30c-43a0-ba2e-9dcc187b6b3d"}}
{"log_message":"(I) GET /v1/vaults/ty324ecoq7a7pjg6mzb5nir2ge/items/nzirrhwyemdfsrr4ebeghbohai completed (200: OK)","timestamp":"2022-09-30T08:47:45.8926955Z","level":3,"scope":{"request_id":"7942ecb3-2b86-9cbb-84c9-caad68232b82","jti":"r33ngkrcmnw5y6bzt64qty5qlm"}}

{"log_message":"(I) GET /v1/vaults/ty324ecoq7a7pjg6mzb5nir2ge/items?filter=title+eq+%22secret5%22","timestamp":"2022-09-30T08:47:45.895190226Z","level":3,"scope":{"request_id":"bcf096e4-51f7-4d0b-840a-3e6ecc9aec4e"}}
{"log_message":"(I) GET /v1/vaults/ty324ecoq7a7pjg6mzb5nir2ge/items?filter=title+eq+%22secret5%22 completed (200: OK)","timestamp":"2022-09-30T08:47:45.89986681Z","level":3,"scope":{"request_id":"4cd38b5e-8c1a-99d1-b51d-dd855fbd5e58","jti":"r33ngkrcmnw5y6bzt64qty5qlm"}}
{"log_message":"(E) 400: Invalid Item UUID","timestamp":"2022-09-30T08:47:45.900104332Z","level":1,"scope":{"request_id":"ffbf6136-0039-9c84-8e2e-3918761b522d","jti":"r33ngkrcmnw5y6bzt64qty5qlm"}}
{"log_message":"(I) GET /v1/vaults/ty324ecoq7a7pjg6mzb5nir2ge/items/ejuchotr3nwsrdolpovizpos74","timestamp":"2022-09-30T08:47:45.90333296Z","level":3,"scope":{"request_id":"e8ffb5cf-a162-4669-a2d6-efdb32f1ae1b"}}
{"log_message":"(I) GET /v1/vaults/ty324ecoq7a7pjg6mzb5nir2ge/items/ejuchotr3nwsrdolpovizpos74 completed (200: OK)","timestamp":"2022-09-30T08:47:45.912064785Z","level":3,"scope":{"request_id":"da1d538b-2508-99d4-b7a6-214e2dc3059d","jti":"r33ngkrcmnw5y6bzt64qty5qlm"}}

{"log_message":"(I) GET /v1/vaults/ty324ecoq7a7pjg6mzb5nir2ge/items?filter=title+eq+%22secret6%22","timestamp":"2022-09-30T08:47:45.914141925Z","level":3,"scope":{"request_id":"6c3c5642-674b-43b6-a82c-484e1a9f20f1"}}
{"log_message":"(I) GET /v1/vaults/ty324ecoq7a7pjg6mzb5nir2ge/items?filter=title+eq+%22secret6%22 completed (200: OK)","timestamp":"2022-09-30T08:47:45.919736084Z","level":3,"scope":{"request_id":"3a725739-c7bb-9b7d-94b3-f50a595adac8","jti":"r33ngkrcmnw5y6bzt64qty5qlm"}}
{"log_message":"(E) 400: Invalid Item UUID","timestamp":"2022-09-30T08:47:45.920372735Z","level":1,"scope":{"request_id":"ffbf6136-0039-9c84-8e2e-3918761b522d","jti":"r33ngkrcmnw5y6bzt64qty5qlm"}}
{"log_message":"(I) GET /v1/vaults/ty324ecoq7a7pjg6mzb5nir2ge/items/uwgmwqs6rqchn75v6rmodifujm","timestamp":"2022-09-30T08:47:45.921751532Z","level":3,"scope":{"request_id":"34e64d58-a2bc-4608-85cd-7530cbbf10a3"}}
{"log_message":"(I) GET /v1/vaults/ty324ecoq7a7pjg6mzb5nir2ge/items/uwgmwqs6rqchn75v6rmodifujm completed (200: OK)","timestamp":"2022-09-30T08:47:45.926003391Z","level":3,"scope":{"request_id":"02497644-c743-9814-882e-76530bfeadc7","jti":"r33ngkrcmnw5y6bzt64qty5qlm"}}

{"log_message":"(I) GET /v1/vaults/ty324ecoq7a7pjg6mzb5nir2ge/items?filter=title+eq+%22secret7%22","timestamp":"2022-09-30T08:47:45.928454364Z","level":3,"scope":{"request_id":"724ac576-07a1-43b9-8d7b-13cbaffa843b"}}
{"log_message":"(I) GET /v1/vaults/ty324ecoq7a7pjg6mzb5nir2ge/items?filter=title+eq+%22secret7%22 completed (200: OK)","timestamp":"2022-09-30T08:47:45.933509008Z","level":3,"scope":{"request_id":"9aa8cc0e-872b-9037-a8b9-0a154d44767e","jti":"r33ngkrcmnw5y6bzt64qty5qlm"}}
{"log_message":"(E) 400: Invalid Item UUID","timestamp":"2022-09-30T08:47:45.933625454Z","level":1,"scope":{"request_id":"ffbf6136-0039-9c84-8e2e-3918761b522d","jti":"r33ngkrcmnw5y6bzt64qty5qlm"}}
{"log_message":"(I) GET /v1/vaults/ty324ecoq7a7pjg6mzb5nir2ge/items/y7dqz7eemg2dsmp4blr47os6ci","timestamp":"2022-09-30T08:47:45.935294542Z","level":3,"scope":{"request_id":"6fe394d2-14f0-4ea0-b71e-0ac4565d9203"}}
{"log_message":"(I) GET /v1/vaults/ty324ecoq7a7pjg6mzb5nir2ge/items/y7dqz7eemg2dsmp4blr47os6ci completed (200: OK)","timestamp":"2022-09-30T08:47:45.939182083Z","level":3,"scope":{"request_id":"98d2c241-7aa4-9454-a4d4-293b090346d4","jti":"r33ngkrcmnw5y6bzt64qty5qlm"}}

{"log_message":"(I) GET /v1/vaults/ty324ecoq7a7pjg6mzb5nir2ge/items?filter=title+eq+%22secret8%22","timestamp":"2022-09-30T08:47:45.941175648Z","level":3,"scope":{"request_id":"6d6ca7fc-f2b1-4b6f-8bcc-e18a5dabfddb"}}
{"log_message":"(I) GET /v1/vaults/ty324ecoq7a7pjg6mzb5nir2ge/items?filter=title+eq+%22secret8%22 completed (200: OK)","timestamp":"2022-09-30T08:47:45.947526138Z","level":3,"scope":{"request_id":"61287fb2-3bdd-9085-99b9-b03d1dd0fac6","jti":"r33ngkrcmnw5y6bzt64qty5qlm"}}
{"log_message":"(E) 400: Invalid Item UUID","timestamp":"2022-09-30T08:47:45.948022069Z","level":1,"scope":{"request_id":"ffbf6136-0039-9c84-8e2e-3918761b522d","jti":"r33ngkrcmnw5y6bzt64qty5qlm"}}
{"log_message":"(I) GET /v1/vaults/ty324ecoq7a7pjg6mzb5nir2ge/items/oslejvrqvcsxt5fwfvgms5kgde","timestamp":"2022-09-30T08:47:45.949733412Z","level":3,"scope":{"request_id":"8e4aa4a4-4525-431f-bfbf-531ffdabf4b2"}}
{"log_message":"(I) GET /v1/vaults/ty324ecoq7a7pjg6mzb5nir2ge/items/oslejvrqvcsxt5fwfvgms5kgde completed (200: OK)","timestamp":"2022-09-30T08:47:45.953466389Z","level":3,"scope":{"request_id":"ec205556-05ae-955c-965b-b8b50a426a92","jti":"r33ngkrcmnw5y6bzt64qty5qlm"}}

[havard024]

Found a similar issue in 1Password/connect repo, linked it above

[ilaif]

This happens to me as well, We also get errors from op CLI with:

[ERROR] 2023/01/09 19:52:24 could not read secret op://xxx/credential: could not retrieve vaults: Get "*****************************************/v1/vaults": dial tcp: lookup op-connect.xxx.xxx on 127.0.0.11:53: server misbehaving

Not sure it's related, but it seems the connect server is not stable.

Any idea?

[YentlFrickx]

This happens to us as well, a lot of "(E) 400: Invalid Item UUID" logs on our PRD cluster

[edif2008]

Hey all.

Thank you for raising this. It looks like the issue is rather related to Connect than the operator.
Therefore, I'd suggest you follow this issue for further updates: 1Password/connect#45

[ninjaMikeG]

@edif2008 , I believe that this is an error in the interaction of Operator with OP Connect. Operator needs to have a different path for injecting by Title - so you can do the proper API call to connect. When using TITLE in the path for a OnePasswordItem, the operator is sending a request passing the title as the ID and not finding it, then querying on the title filter.. This causes constant spamming of bad requests to Connect. The log spam in Connect is misuse of the API by Operator.

When I reference items by ID, the log spamming stops, but when I reference by title it continually dumps these errors (even though the secret creation is successful and correct).