-
Notifications
You must be signed in to change notification settings - Fork 171
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Have an option to verify the locally pulled files #1368
Comments
@stmlange Can you kindly explain your scenario and why the verification is needed? |
@qweeah Thanks for reaching out. The main reason why I have created this ticket is that there doesn't seem to be an (easy) option to verify/check if the locally pulled artifacts are "equal" to what the remove artifact is. Try to answer the question: is what I have locally really what was published to remote? Consider for example maven/gradle that publish dedicated sha1 and md5 hashsums so one can download the hashsum and verify somehow that the published thing is "correct"/"equal" to the local variant. With oras one can have multiple digest variants encoded (https://github.com/opencontainers/image-spec/blob/main/descriptor.md#digests). A digest can be Running oras pull multiple times actually seems to re-download the artifact. So it may be possible that oras pull secretly checks those digests (and potentially fails if the download was not successfull), but redownloading is a waste of network resources. |
Thanks @stmlange for the detailed explanation. /cc @FeynmanZhou To validate if it's something we should add to ORAS CLI. |
@stmlange Also worth mentioning that if you want to copy an artifact in a trusted way, why not using an OCI image layout? # 1. copy an artifact to a local folder mcr.microsoft.com/oss/kubernetes/kubectl
> oras cp mcr.microsoft.com/oss/kubernetes/kubectl:v1.28.1 -r --to-oci-layout mcr.microsoft.com/oss/kubernetes/kubectl
✓ Copied application/vnd.docker.container.image.v1+json 1.93/1.93 kB 100.00% 571µs
└─ sha256:919d96c9446db8f5c6cf76d98abd4c79ccfe9af241f977d87188ef3e9f6f09de
...
Copied [registry] mcr.microsoft.com/oss/kubernetes/kubectl:v1.28.1 => [oci-layout] mcr.microsoft.com/oss/kubernetes/kubectl
Digest: sha256:a01b2873f41c65aa9157baf5ec0e0beaf80e9e84bb7dfa94b081cd230b534418
# 2. cp the OCI image layout folder to some air-gap environment
# 3. pull provenance file from the copied OCI image layout folder, checksum will be verified during the pull
> oras pull --oci-layout mcr.microsoft.com/oss/kubernetes/kubectl@sha256:30019e253ab74eb3e38abae7b8997e8e60c420169
044ca9bfaf9665f54ad18bc -o in-toto
✓ Pulled provenance.json 14.9/14.9 kB 100.00% 717µs
└─ sha256:f4740e5a3adde42224679263c7b4e76985411cb7a9504615cf1421d8afb078b5
✓ Pulled application/vnd.oci.image.manifest.v1+json 682/682 B 100.00% 608µs
└─ sha256:30019e253ab74eb3e38abae7b8997e8e60c420169044ca9bfaf9665f54ad18bc
Pulled [oci-layout] mcr.microsoft.com/oss/kubernetes/kubectl@sha256:30019e253ab74eb3e38abae7b8997e8e60c420169044ca9bfaf9665f54ad18bc
Digest: sha256:30019e253ab74eb3e38abae7b8997e8e60c420169044ca9bfaf9665f54ad18bc |
Indeed with ORAS the manual way would be to download the manifest (e.g. I believe as per https://github.com/opencontainers/image-spec/blob/main/descriptor.md#digests oras could also have a sha512:401b09eab3c013d4ca54922bb802bec8fd5318192b0a75f201d8b372742 or a hashdigest of |
I am not an expert of checksum file but shouldn't the length of the checksum string implies the algorithm already? |
Yes I tested on my linux VM and different checksum can co-exist in the same checksum file > cat a
123
> shasum -a 512 a >> sum
> shasum -a 256 a >> sum
> cat sum
ea2fe56bb8c1fb5ada84963b42ed71b764a74b092d75755173ade06f2f4aada9c00d6c302e185035cbe85fdff31698bca93e8661f0cbcef52cf2ff65864fd742 a
181210f8f9c779c26da1d9b2075bde0127302ee0e3fca38c9a83f5b1dd8e5d3b a
> shasum -c sum
a: OK
a: OK |
What I mean is although ORAS doesn't support sha512, still you may split the digest with |
Yes in general the length of the hashed string could be used to determine the algorithm. The ORAS Digest (https://github.com/opencontainers/image-spec/blob/main/descriptor.md#digests) can be more than just |
The demo I give generates sha256sum and sha512sum into on |
@stmlange You don't need to use |
The problem remains that ORAS can encode the hash as Consider the Consider:
|
|
Ok I see that only However I still think it is not that easy (I guess sometimes even impossible) to run a It just tells us the "digest", but we don't know the filename.
For a shasum to work we need both filename and digest:
in theory one could workaround the issue by attaching the filenames using annotation to the manifest like:
I still think and feel that manual validation is not the way to go :-) |
Generating the checksum file is not easy with v1.1.0 but will be improved in v1.2.0. You can try the main build container, e.g. generate checksum file for > docker run ghcr.io/oras-project/oras:main manifest fetch mcr.microsoft.com/oss/kubernetes/kubectl@sha256:30019e253ab74eb3e38abae7b8997e8e60c420169044ca9bfaf9665f54ad18bc --format '{{range .content.layers}}{{if index .annotations "org.opencontainers.image.title"}}{{.digest}} {{index .annotations "org.opencontainers.image.title"}}{{println}}{{end}}{{end}}'
sha256:f4740e5a3adde42224679263c7b4e76985411cb7a9504615cf1421d8afb078b5 provenance.json |
This issue is stale because it has been open 60 days with no activity. Remove stale label or comment or this will be closed in 30 days. |
This issue was closed because it has been stalled for 30 days with no activity. |
What is the version of your ORAS CLI
1.1.0
What would you like to be added?
Maybe I'm missing it, but assume I have a local copy of some pulled data. Is there an option to run some sha256check or something to verify that the local copy matches with the artifacts listed in the manifest?
Why is this needed for ORAS?
The manifest can have multiple digest encoding's which can make it very tricky to manually verify if the local copy is what is equal to the remote artifact.
Are you willing to submit PRs to contribute to this feature?
The text was updated successfully, but these errors were encountered: