Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Adding retry logic to the TritonInferenceStage to allow recovering from errors #1548

Merged
merged 69 commits into from
Apr 3, 2024

Conversation

cwharris
Copy link
Contributor

@cwharris cwharris commented Mar 6, 2024

Factor out the Triton inference logic in the Triton inference stage to a client class which can be swapped out for a fake/test client. Keep retry logic in the Stage and test using a fake/test client.

Closes #1510

By Submitting this PR I confirm:

  • I am familiar with the Contributing Guidelines.
  • When the PR is ready for review, new or existing tests cover these changes.
  • When the PR is ready for review, the documentation is up to date with these changes.

@cwharris cwharris added non-breaking Non-breaking change feature request New feature or request labels Mar 6, 2024
@cwharris cwharris marked this pull request as ready for review March 12, 2024 16:57
@cwharris cwharris requested a review from a team as a code owner March 12, 2024 16:57
@mdemoret-nv mdemoret-nv changed the title Triton inf refactor Adding retry logic to the TritonInferenceStage to allow recovering from errors Mar 27, 2024
@cwharris cwharris requested a review from a team as a code owner April 1, 2024 17:38
@mdemoret-nv
Copy link
Contributor

/merge

@rapids-bot rapids-bot bot merged commit 10922a4 into nv-morpheus:branch-24.03 Apr 3, 2024
10 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
feature request New feature or request non-breaking Non-breaking change
Projects
Archived in project
2 participants