Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Don't set pe_count for the C++ impl of the TritonInferenceStage #1640

Merged
merged 13 commits into from
Apr 19, 2024
Merged

Don't set pe_count for the C++ impl of the TritonInferenceStage #1640

merged 13 commits into from
Apr 19, 2024

Conversation

dagardner-nv
Copy link
Contributor

@dagardner-nv dagardner-nv commented Apr 18, 2024 •
edited
Loading

Description

Closes #1639

By Submitting this PR I confirm:

  • I am familiar with the Contributing Guidelines.
  • When the PR is ready for review, new or existing tests cover these changes.
  • When the PR is ready for review, the documentation is up to date with these changes.

@dagardner-nv dagardner-nv self-assigned this Apr 18, 2024
@dagardner-nv dagardner-nv requested a review from a team as a code owner April 18, 2024 19:29
@dagardner-nv dagardner-nv added bug Something isn't working non-breaking Non-breaking change ! - Hotfix Related to a Hotfix Release labels Apr 18, 2024
@dagardner-nv dagardner-nv changed the base branch from branch-24.06 to branch-24.03 April 18, 2024 19:30
@dagardner-nv dagardner-nv marked this pull request as draft April 18, 2024 19:30
mdemoret-nv
mdemoret-nv requested changes Apr 18, 2024
View reviewed changes
Copy link
Contributor

@mdemoret-nv mdemoret-nv left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Couple of suggestions. Big things:

  • Why are we only checking the results with 1 thread? It should work for all
  • Use os.cpu_count() instead of hard coding to 4 threads

morpheus/stages/inference/inference_stage.py Outdated Show resolved Hide resolved
tests/test_abp.py Outdated Show resolved Hide resolved
tests/test_abp.py Outdated Show resolved Hide resolved
tests/test_abp.py Outdated Show resolved Hide resolved
tests/test_sid.py Outdated Show resolved Hide resolved
@dagardner-nv
Revert "Remove the launch options entirely since we currently don't h…
ee03438 
…ave a reliable way to perform a multi-threaded triton unittest"

This reverts commit 8444f77.
@dagardner-nv
Revert "Only set the pe_count for python impls"
a0092c7 
This reverts commit 8f6209a.
@dagardner-nv dagardner-nv changed the title Don't set pe_count for inference stages with a C++ impl Don't set pe_count for Triton inference stages Apr 18, 2024
@dagardner-nv dagardner-nv marked this pull request as ready for review April 18, 2024 21:23
@dagardner-nv dagardner-nv changed the title Don't set pe_count for Triton inference stages Don't set pe_count for the C++ impl of the TritonInferenceStage Apr 18, 2024
@dagardner-nv dagardner-nv mentioned this pull request Apr 18, 2024
Merged
@efajardo-nv @dagardner-nv
Use conda env create --yes instead of --force (nv-morpheus#1636)
9f09023 
conda dropped support for the --force flag to conda env create. This changes that flag name to --yes.
See https://github.com/conda/conda/blob/main/CHANGELOG.md#2430-2024-03-12 and rapidsai/miniforge-cuda#63 for more info.

## By Submitting this PR I confirm:
- I am familiar with the [Contributing Guidelines](https://github.com/nv-morpheus/Morpheus/blob/main/docs/source/developer_guide/contributing.md).
- When the PR is ready for review, new or existing tests cover these changes.
- When the PR is ready for review, the documentation is up to date with these changes.

Authors:
  - Eli Fajardo (https://github.com/efajardo-nv)

Approvers:
  - David Gardner (https://github.com/dagardner-nv)

URL: nv-morpheus#1636
@dagardner-nv
Copy link
Contributor Author

/merge

@rapids-bot rapids-bot bot merged commit 883b804 into nv-morpheus:branch-24.03 Apr 19, 2024
12 of 13 checks passed
@dagardner-nv dagardner-nv deleted the david-triton-sid-threading branch April 19, 2024 20:43
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@mdemoret-nv mdemoret-nv mdemoret-nv approved these changes

Assignees

@dagardner-nv dagardner-nv

Labels
! - Hotfix Related to a Hotfix Release bug Something isn't working non-breaking Non-breaking change
Projects
Morpheus Boards
Archived in project
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.

[BUG]: SID example crashes at Triton inference stage (24.03.01 runtime image)
3 participants
@dagardner-nv @mdemoret-nv @efajardo-nv