-
Notifications
You must be signed in to change notification settings - Fork 7
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Load all timesteps for a given analysis_assim_extend reference time #156
Comments
@samlamont @mgdenno Has this issue been resolved? I need to load an additional day of data for Hurricane Debby and wondering if it's possible or if I have to reload the entire period to get the complete AnA time series without gaps. |
Unfortunately, no, it hasn't be addressed yet. I thought it had, but Sam and I reviewed open issues last week and this has not been resolved. Just off the top of my head, one workaround besides redownloading all data might be to write to a different location and then give conflicting files unique names when moving the new data to the existing data directory. |
Hi @kvanwerkhoven apologies for the delay on this. Just to confirm, we want the fetching of NWM analysis data to behave like any other forecast (prioritizing reference_time)? So if one day is requested for a single location for the standard analysis configuration ( |
Yes, I think either eliminate |
I keep running into issues with missing data due to the behavior of the analysis_assim loading function, which loads partial files for the first and last dates of a requested range (i.e., the values times versus reference times within the requested range). I frequently load portions of a period and then add a few days or reload part of the period for more locations, etc. I am constantly inadvertently overwriting complete files with partial files as a result. I believe the default expected behavior should be to always load all timesteps for a given dataset. Thus if a parquet file exists with a given reference timestamp, one can expect that it includes all the data for that reference time. If there was a reason for the opposite behavior, let's discuss.
The text was updated successfully, but these errors were encountered: