-
Notifications
You must be signed in to change notification settings - Fork 50
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Validating inference framework image tags #357
Conversation
51c9f8c
to
bc15105
Compare
thanks for the contributions! i'm worried about the overhead of adding these tags / redeploy model engine server whenever we're adding a new image. possible to query ECR and check existence of the image? |
hm I see that's a good suggestion, I'll give it a try and see if it works! |
dca91d0
to
6ba1f02
Compare
@yunfeng-scale I implemented querying ECR -- what are your thoughts? |
model-engine/model_engine_server/domain/use_cases/llm_model_endpoint_use_cases.py
Show resolved
Hide resolved
model-engine/model_engine_server/domain/use_cases/llm_model_endpoint_use_cases.py
Outdated
Show resolved
Hide resolved
it seems a some commits were reverted, can you bring them back? |
model-engine/model_engine_server/domain/use_cases/llm_model_endpoint_use_cases.py
Outdated
Show resolved
Hide resolved
model-engine/model_engine_server/domain/use_cases/llm_model_endpoint_use_cases.py
Outdated
Show resolved
Hide resolved
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
please fix @yixu34 's comment
Added unit test to this: #338
Needed to rebase so created a separate PR.
Querying ECR to check for existing images for each inference framework.