Add Azure Signer support #588

malancas · 2023-05-26T14:05:25Z

Fixes: #

Description of the changes being introduced by the pull request:

This pull request adds an Azure signer implementation so we can use Azure's KMS offering Key Vault. This signer currently only supports ECDSA keys. RSA key support can be added later in a follow up pull request.

Please verify and check that the pull request fulfils the following requirements:

The code follows the Code Style Guidelines
Tests have been added for the bug fix or new feature
Docs have been added for the bug fix or new feature

Signed-off-by: Meredith Lancaster <[email protected]>

Signed-off-by: Fredrik Skogman <[email protected]>

Added packages to requirements and a small bug fix

Signed-off-by: Fredrik Skogman <[email protected]>

Added missing imports

Signed-off-by: Meredith Lancaster <[email protected]>

The CryptographySigner expectes the digest when it performs a signing operation. It's a nicer API to let the AzureSigner calculate the digest.

Signed-off-by: Fredrik Skogman <[email protected]>

Make sure to calculated the digest using the correct hash function.

Signed-off-by: Meredith Lancaster <[email protected]>

…rmat. Signed-off-by: Fredrik Skogman <[email protected]>

Signed-off-by: Fredrik Skogman <[email protected]>

Add import method

Signed-off-by: Meredith Lancaster <[email protected]>

jku · 2023-05-26T15:31:32Z

Based on a quick glance, looks nice!
Some early comments:

tests are failing because the signer modules should be importable even without the optional deps (and cryptography is an optional dep -- you could probably move the import inside the try-except you already have) -- I fixed this one in a later commits
tox -e lint runs the same lint tests locally as the CI runs. Please use it, ask if the results don't seem to make sense
I think the import_() should not take the fully built "private key URI" as argument -- that forces the caller to understand and be able to build the URI. Instead it should take the logical arguments that azure vault users will understand and the import method should build the URI

Finally a suggestion/comment, feel free to disregard:

from experience with GCP, I would suggest you might want to use the full azure keyid that contains the vault and key version, if that is easily available to users (because we likely do want a specific key version): https://myvault.vault.azure.net/keys/my-key/my-key-version,
- this full keyid might work as the import argument
- you probably won't need to call KeyClient() during __init__() since you can use the full keyid to create CryptographyClient: https://learn.microsoft.com/en-us/python/api/azure-keyvault-keys/azure.keyvault.keys.crypto.cryptographyclient?view=azure-python

Signed-off-by: Meredith Lancaster <[email protected]>

Specifically $ black . $ isort .

These are the trivial looking issues: there are still some left that require actual decisions (both in pylint and mypy)

To allow the _azure_signer module import when azure dependency modules are not installed, use stringized annotations (this allows method definitions to happen without the types existing, while still letting the static type checkers do their work).

jku · 2023-05-29T11:54:22Z

Since you're allowing maintainer pushes (thanks!), I'll take the liberty of adding some annotation and lint fixes:

most changes come from running black and isort (like tox -e lint does) but there are some manual lint fixes and some annotation tweaks as well
I've only touched things that should not affect intended function
the unrelated tests should pass after this
there are still linter issues (both pylint and mypy) that require actual changes

Obviously feel free to remove my commits if you we're already working on this and don't need them

It's clear now wheter it's a private uri (azurekms) or an azure uri (https) Signed-off-by: Fredrik Skogman <[email protected]>

Cleaned up the usage of uris.

Signed-off-by: Meredith Lancaster <[email protected]>

…lib into add-azure-support

… too Signed-off-by: Fredrik Skogman <[email protected]>

Update the import method to return a key id that contains the version…

jku · 2023-05-30T12:50:59Z

looks good! WRT linter failures:

there are some more from pylint and mypy (the tox -e lint run just ends on first error)
IIRC most of them are about some azure class not having attributes -- totally fine to skip those checks somehow if that seems appropriate

Signed-off-by: Meredith Lancaster <[email protected]>

jku · 2023-05-30T13:16:59Z

IIRC most of them are about some azure class not having attributes

scratch that comment: these were seemingly all in the code that you managed to remove in the last commits

Signed-off-by: Meredith Lancaster <[email protected]>

malancas · 2023-05-30T13:28:09Z

@jku All of the linting errors should now be fixed. Thanks for your feedback and help with this PR. I did disable pylint for the too-many-locals rule that was failing on the import_ class method because I find it clearer to use all of the local variables declared in that method. But let me know otherwise.

kommendorkapten · 2023-05-30T15:21:50Z

We can take this out of "draft" now @malancas right?

malancas · 2023-05-30T15:29:49Z

@kommendorkapten yep, I'll change that now.

lukpueh

Great work, @malancas!

securesystemslib/signer/_azure_signer.py

lukpueh · 2023-05-31T07:49:24Z

securesystemslib/signer/_azure_signer.py

+                public_key,
+            )
+            self.hash_algorithm = self._get_hash_algorithm(public_key)
+        except UnsupportedKeyType as e:


Should UnsupportedKeyType be public API? Currently it isn't, so a user can't really handle it. Would a built-in exception type also do the trick?

Also, is it really worth to catch, log and re-raise? Doesn't the original error message, which you control, provide enough info? This pattern seems more useful below, where you contextualise the 3rd-party HttpResponseError, although even there I wonder if the stack trace alone might be enough. 🤷

FYI: we have a discussion ticket about Signer exceptions in #468. Curious about your opinions.

I left a note about this: #588 (comment)

TLDR: I think it's not really public API, it's just part of the larger group of exceptions that we know we may raise that callers are not really meant to handle: GCPSigner handles similar cases by just allowing KeyErrors from dict lookups go through...

Agreed. I was actually trying to challenge the need for a custom exception and the way it is handled internally. But I'm okay with it. The code is still very readable and extra info for the caller is nice.

jku

I think the code looks good. left some notes but nothing that I think requires changes if you'd like to merge as is.

The one thing I'd like to bring up is testing... I know it's tricky to setup a proper test with the KMS service dependency (I talk a bit more about that in a comment) and we can't require a real time testing setup. But right now we as maintainers are in a situation where we can't see

example code -- how is it supposed to be used
proof that it actually works anywhere -- logs of a successful signing

Do you have good ideas about this?
Are there public logs of this working?
Should we add testing code (similar to check_kms_signers) even if it is disabled and only works in your fork?

requirements-kms.txt

securesystemslib/signer/_azure_signer.py

Signed-off-by: Fredrik Skogman <[email protected]>

kommendorkapten · 2023-05-31T12:40:52Z

I'll work on adding azure support in check_kms_signers, I can share credentials with you if you want to replicate the test locally.

jku · 2023-05-31T13:11:11Z

I can share credentials with you if you want to replicate the test locally.

Oh no need for that -- would just be good to know that if someone needs to test this, they mostly just need to setup the KMS (and maybe enable the tests or run another tox env or something simple like that).

Co-authored-by: Jussi Kukkonen <[email protected]>

Signed-off-by: Meredith Lancaster <[email protected]>

…ify signature creation Signed-off-by: Meredith Lancaster <[email protected]>

Added smoke tests for Azure KMS sign. Signed-off-by: Fredrik Skogman <[email protected]>

jku

👍 thanks for the test: it works exactly like I expected but it's good to actually see it.

The requirements-kms.txt changes are now not needed (since it is used as requirements for tests.check_kms_signers which doesn't currently include azure). I can remove those changes before merging.

requirements-kms.txt lists the requirements to run tests.check_kms_signers (which is run on CI): Azure is not currently part of that test set so the requirements are not needed either.

malancas · 2023-06-01T16:01:47Z

Great, thank you for the reviews!

malancas and others added 19 commits May 23, 2023 19:40

start adding azure signer implementation

f6ffa19

Signed-off-by: Meredith Lancaster <[email protected]>

Added packages to requirements and a small bug fix

618b4de

Signed-off-by: Fredrik Skogman <[email protected]>

Removed debub print

a5317d1

Signed-off-by: Fredrik Skogman <[email protected]>

Merge pull request #1 from kommendorkapten/fixes

1502d94

Added packages to requirements and a small bug fix

Added missing imports

c88c7f9

Signed-off-by: Fredrik Skogman <[email protected]>

Use the keyvault uri

ba237b8

Merge pull request #2 from kommendorkapten/fix_imports2

90f15a4

Added missing imports

replace match with if else

8fe44e7

Signed-off-by: Meredith Lancaster <[email protected]>

working signer creation and signing

3239a7e

Signed-off-by: Meredith Lancaster <[email protected]>

clean up functions

d7437bd

Signed-off-by: Meredith Lancaster <[email protected]>

Make sure to calculated the digest using the correct hash function.

df9b5c2

The CryptographySigner expectes the digest when it performs a signing operation. It's a nicer API to let the AzureSigner calculate the digest.

Make sure signature is in ASN.1 format

a820bcf

Signed-off-by: Fredrik Skogman <[email protected]>

Merge pull request #4 from kommendorkapten/sign_digest

942a229

Make sure to calculated the digest using the correct hash function.

add comments and exceptions around only supporting ec keys

041be38

Signed-off-by: Meredith Lancaster <[email protected]>

Added an import method with returns the public key in the expected fo…

473648c

…rmat. Signed-off-by: Fredrik Skogman <[email protected]>

Updated comment to be correct on second ctor parameter

41a89ae

Signed-off-by: Fredrik Skogman <[email protected]>

Merge pull request #5 from kommendorkapten/add_import_method

ca0dad4

Add import method

Merge remote-tracking branch 'upstream/main' into add-azure-support

1ac6bc9

remove extra staticmethod

4184d0b

Signed-off-by: Meredith Lancaster <[email protected]>

malancas and others added 4 commits May 26, 2023 10:51

move cryptography imports

f2c354f

Signed-off-by: Meredith Lancaster <[email protected]>

lint: Run black and isort

fe326b0

Specifically $ black . $ isort .

AzureSigner: Fix a bunch of lint issues

a692a5e

These are the trivial looking issues: there are still some left that require actual decisions (both in pylint and mypy)

Cleaned up the usage of uris.

64a536e

It's clear now wheter it's a private uri (azurekms) or an azure uri (https) Signed-off-by: Fredrik Skogman <[email protected]>

kommendorkapten mentioned this pull request May 30, 2023

Added azure kms support jku/repository-playground#89

Merged

malancas and others added 3 commits May 30, 2023 06:07

Merge pull request #8 from kommendorkapten/uri_fixes

4272655

Cleaned up the usage of uris.

remove test runner

d6e60b6

Signed-off-by: Meredith Lancaster <[email protected]>

Merge branch 'add-azure-support' of github.com:malancas/securesystems…

a702b39

…lib into add-azure-support

kommendorkapten and others added 2 commits May 30, 2023 14:17

Update the import method to return a key id that contains the version…

80bf9a2

… too Signed-off-by: Fredrik Skogman <[email protected]>

Merge pull request #9 from kommendorkapten/az_use_key_ver

8fb25cc

Update the import method to return a key id that contains the version…

malancas added 2 commits May 30, 2023 06:59

fix linting issues

9232e44

Signed-off-by: Meredith Lancaster <[email protected]>

linter fix, raise error after logging

960b741

Signed-off-by: Meredith Lancaster <[email protected]>

malancas added 2 commits May 30, 2023 07:23

add docstrings

688555e

Signed-off-by: Meredith Lancaster <[email protected]>

add pylint disable comment for too-many-locals

c00d3df

Signed-off-by: Meredith Lancaster <[email protected]>

malancas marked this pull request as ready for review May 30, 2023 15:29

lukpueh reviewed May 31, 2023

View reviewed changes

jku reviewed May 31, 2023

View reviewed changes

kommendorkapten and others added 2 commits May 31, 2023 14:25

Support all EC keys, not just HSM

3c99967

Signed-off-by: Fredrik Skogman <[email protected]>

Merge pull request #10 from kommendorkapten/all_ec_keys

b17370e

malancas and others added 3 commits May 31, 2023 07:21

More actionable error message

2d7c05b

Co-authored-by: Jussi Kukkonen <[email protected]>

pr feedback, don't need to stringify Key

f540090

Signed-off-by: Meredith Lancaster <[email protected]>

pr feedback, add section with roles needed by the azure signer, simpl…

e905b27

…ify signature creation Signed-off-by: Meredith Lancaster <[email protected]>

lukpueh mentioned this pull request May 31, 2023

signer: deduplicate signing scheme dissection #594

Open

kommendorkapten and others added 2 commits June 1, 2023 12:44

Fixed a bug that sets the wrong key id on the returned signature.

ca60489

Added smoke tests for Azure KMS sign. Signed-off-by: Fredrik Skogman <[email protected]>

Merge pull request #11 from kommendorkapten/correct_key_id

eeabc68

jku approved these changes Jun 1, 2023

View reviewed changes

requirements-kms: Remove azure packages

36b65a4

requirements-kms.txt lists the requirements to run tests.check_kms_signers (which is run on CI): Azure is not currently part of that test set so the requirements are not needed either.

jku merged commit a4a86db into secure-systems-lab:main Jun 1, 2023

MVrachev mentioned this pull request Aug 30, 2023

Release v0.29.0 #631

Closed

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Add Azure Signer support #588

Add Azure Signer support #588

malancas commented May 26, 2023

jku commented May 26, 2023 •

edited

Loading

jku commented May 29, 2023 •

edited

Loading

jku commented May 30, 2023

jku commented May 30, 2023

malancas commented May 30, 2023 •

edited

Loading

kommendorkapten commented May 30, 2023

malancas commented May 30, 2023

lukpueh left a comment

lukpueh May 31, 2023

jku May 31, 2023 •

edited

Loading

lukpueh May 31, 2023

jku left a comment

kommendorkapten commented May 31, 2023

jku commented May 31, 2023

jku left a comment

malancas commented Jun 1, 2023

Add Azure Signer support #588

Add Azure Signer support #588

Conversation

malancas commented May 26, 2023

Description of the changes being introduced by the pull request:

Please verify and check that the pull request fulfils the following requirements:

jku commented May 26, 2023 • edited Loading

jku commented May 29, 2023 • edited Loading

jku commented May 30, 2023

jku commented May 30, 2023

malancas commented May 30, 2023 • edited Loading

kommendorkapten commented May 30, 2023

malancas commented May 30, 2023

lukpueh left a comment

Choose a reason for hiding this comment

lukpueh May 31, 2023

Choose a reason for hiding this comment

jku May 31, 2023 • edited Loading

Choose a reason for hiding this comment

lukpueh May 31, 2023

Choose a reason for hiding this comment

jku left a comment

Choose a reason for hiding this comment

kommendorkapten commented May 31, 2023

jku commented May 31, 2023

jku left a comment

Choose a reason for hiding this comment

malancas commented Jun 1, 2023

jku commented May 26, 2023 •

edited

Loading

jku commented May 29, 2023 •

edited

Loading

malancas commented May 30, 2023 •

edited

Loading

jku May 31, 2023 •

edited

Loading