-
Notifications
You must be signed in to change notification settings - Fork 0
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. Weβll occasionally send you account related emails.
Already on GitHub? Sign in to your account
As a user, I want to skip upload of files that are already in the Registry #99
Comments
from the breakout meeting today: file name/path We are not sure yet what the best key should provided by the API, either: option1: node + file path β returns md5sum, lidvid With option 1, a new end-point for the API could be: /files/{node}/{file_path} which would return the {md5sum}Β The issue is that the file_path is not always the same, on the staging bucket or where the file is archived eventually.Β @ramesh-maddegoda, @viviant100 could you investigate how the path on the archive bucket (ODR) is being created from the path in the staging bucket ? |
As discussed today, I will create a ticket to have an end-point in the api: We'll make that part of the Registry API. |
@collinss-jpl can you validate that what is above works for you ? |
@tloubrieu-jpl Yes I think that would work. Does the Registry API use API Gateway though? Will the DUM client need to provide an authentication token with the request to the new endpoint? |
@collinss-jpl @tloubrieu-jpl just want to check on a status for this? has this been implemented and can it be tested at least locally? For DUM, we should just make sure we throw a warning when the API is down, but then keep going through the processing so we aren't blocking the workflow when we have system downtime. |
Checked for duplicates
Yes - I've already checked
π§βπ¬ User Persona(s)
Node Operator
πͺ Motivation
...so that I do not try to reload the data
π Additional Details
No response
Acceptance Criteria
Given
When I perform
Then I expect
βοΈ Engineering Details
The easiest way to do this would be search the registry either for the file path OR by checksum OR both? We could do this with the LID/LIDVID but I think that will add some significant overhead.
Do we want to figure out some sort of auto-generated UUID for every file we upload to the cloud and add this as metadata? Maybe this is something we could actually store then in the Nucleus database and eventually in the registry. It could link throughout the whole system, agnostic of the LIDVID for the products themselves.
The text was updated successfully, but these errors were encountered: