Skip to content

Commit

Permalink
Apply PR suggestions from Mike McKiernan
Browse files Browse the repository at this point in the history
  • Loading branch information
dagardner-nv committed Jul 26, 2024
1 parent 24cda33 commit c1b30a1
Show file tree
Hide file tree
Showing 2 changed files with 4 additions and 4 deletions.
6 changes: 3 additions & 3 deletions docs/source/developer_guide/architecture.md
Original file line number Diff line number Diff line change
Expand Up @@ -63,11 +63,11 @@ The stage identifier is a unique string used in both logging and creating the st

### Type Inference

To perform work, each stage needs to know what type of data it will be operating on. Since Morpheus can pass any type of data from stage to stage, the pipeline must ensure compatible types at every edge connection between stages. This process is called stage type inference and is performed during the pipeline build phase.
To perform work, each stage needs to know what type of data it will be operating on. Morpheus can pass any type of data from stage to stage, the pipeline must ensure compatible types at every edge connection between stages. This process is called stage type inference and is performed during the pipeline build phase.

Stage type inference is necessary because the output type of some stages may depend on the output type of the previous stage. For example, consider a simple pass through stage that passes the input message to the next stage unmodified. If our pass through stage is preceded by a stage generating a string, its output type will be a string. Instead, if it's preceded by a stage generating an integer, its output type will be an integer.
Stage type inference is necessary because the output type of some stages can depend on the output type of the previous stage. For example, consider a simple pass through stage that passes the input message to the next stage unmodified. If our pass through stage is preceded by a stage generating a string, its output type will be a string. Instead, if it's preceded by a stage generating an integer, its output type will be an integer.

Due to the dynamic nature of the output type of a stage, stages must specify a type inference function that accepts an input type and returns the output type. Starting at the source stages, the pipeline will use this function to determine the output type of the source stages. This result will then be passed to the type inference function of the next stage, until the input and output types of every stage in the pipeline have been determined.
Due to the dynamic nature of the output type of a stage, stages must specify a type inference function that accepts an input type and returns the output type. Starting at the source stages, the pipeline will use this function to determine the output type of the source stages. This result is then be passed to the type inference function of the next stage, until the input and output types of every stage in the pipeline have been determined.

After the build phase, the output types of stages cannot be changed. Returning a different type than specified during the build phase will result in undefined behavior.

Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -164,7 +164,7 @@ There are a number of modules that are used in both the training and inference p

Source: `examples/digital_fingerprinting/production/morpheus/dfp/modules/dfp_preproc.py`

The `dfp_preproc` module is a functional component within the Morpheus framework that combines multiple data filtering and processing pipeline modules related to inference and training. This module simplifies the pipeline by consolidating various modules into a single, cohesive unit. The `dfp_preproc` module allows for the configuration of parameters such as the cache directory, timestamp column name, pre-filter options, batching options, user splitting options, and supported data loaders for various file types.
The `dfp_preproc` module is a functional component within the Morpheus framework that combines multiple data filtering and processing pipeline modules related to inference and training. This module simplifies the pipeline by consolidating various modules into a single, cohesive unit. The `dfp_preproc` module supports configuration parameters such as the cache directory, timestamp column name, pre-filter options, batching options, user splitting options, and supported data loaders for various file types.

The module itself consists of a series of chained sub-modules, which are connected in a logical sequence:

Expand Down

0 comments on commit c1b30a1

Please sign in to comment.