Surfacing Morpheus to users / Additional PCAP models

[gary-sixgen]

I've managed to automate the installation of Morpheus and have a system that generates a large amount of PCAP data, but now...

1. Enabling a user to use Morpheus via a UI? Anyone have a PoC out there? It seems that the UI might be tailored based on specific models being used. The SID model seems a good one to use. (Edit: I see this https://github.com/nv-morpheus/morpheus-visualizations/tree/branch-23.11#2-morpheus-graph-visualization-tool)

2. Are there other PCAP based models in development? Any good guides for developing our own models?

[mdemoret-nv]

Enabling a user to use Morpheus via a UI? Anyone have a PoC out there? It seems that the UI might be tailored based on specific models being used

Can you give me more information about how the UI would be used and how it would interact with Morpheus? Are you referring to Morpheus showing a UI using the same process as the pipeline? Or Morpheus taking commands from a UI in a client/server architecture? Or possibly something else? There are many ways that Morpheus could be used via a UI and any additional information you can provide will be helpful.

As far as PoC that already exist, the following items come to mind:

1. Morpheus showing a UI to display output from a pipeline
   1. We have the morpheus-visualizations repo which contains any visualizations/demos that have been created to take the output from Morpheus and generate some sort of visualization (both interactive and non-interactive). In that repo we currently have the following demos:
      1. SID visualization
         1. This demo shows an interactive view of the detections that come out of the SID workflow and displays them in a graph
      2. DFP visualization
         1. This demo shows an overview of the detections from the DFP workflow that give an overall status of how the system is performing
2. Morpheus using a UI to control a pipeline or generate inputs
   1. Part of the DFP example contains a demo that uses a small web app to generate input control messages for a pipeline allowing users to dynamically insert messages into a running pipeline
   2. Some prototyping has been done using python libraries to show a UI from a Morpheus stage without any issues. It should be possible to use libraries such TkInter or PyQt to show a UI right from Morpheus.

Let me know if any of that is similar to what you were thinking.

[gary-sixgen]

This was very helpful.

Can you give me more information about how the UI would be used and how it would interact with Morpheus? Are you referring to Morpheus showing a UI using the same process as the pipeline? Or Morpheus taking commands from a UI in a client/server architecture? Or possibly something else? There are many ways that Morpheus could be used via a UI and any additional information you can provide will be helpful.

As the number of possible pre-trained models grow, I imagine having a UI available to user to switch what model they're using would be helpful. Specifically, I'm deploying via kubernetes and got to the Model Deployment section. It would be easy enough to pre-automate one of the models, but I'd like to surface that as configurable for the users and enable the ability for "runtime" changes to what model is current deployed. (This begs the question of whether I could have multiple models deployed at once...?)

I did imagine each model needs it's own UI (like you mentioned with the visualizations). At this time, I'm explicitly interested in the pre-trained models and presenting those to users via a web interface.

After some digging, I found a lot of those resources you listed, although there are some undiscovered things I'll be looking into.

Thank you for the timely response. :)

[mdemoret-nv]

Thanks for the background information. Its helpful in understanding your use case. While I dont think we have anything that exactly matches what you are trying to do, there are a few good options:

1. If you are planning on using Triton to handle the inference requests, its possible to load/unload models while a pipeline is running without the need to modify Morpheus in any way. Once the model has been changed, any subsequent request sent by Morpheus will use the new model. If you go down this route, there are a couple of things to consider:
   1. Loading/unloading models can be controlled by the Triton REST/gRPC API. This allows loading/unloading models with something like: curl --request POST --url http://localhost:8000/v2/repository/models/sid-minibert-trt/load. This could easily be wrapped in a UI if needed.
   2. If you go down this route, just make sure to always have a model with the right name loaded at all times. There are some tricks that would be required to guarantee this, so let me know if this is the route you go down.
2. If you dont use Triton, you can use ML Flow to host your models which supports model versioning and has a UI that allows you to control which model should be used dynamically.

   1. We use this functionality for our Digital Fingerprinting workflow so it has been tested and works really well. The UI allows you to select which version of a model to use for production which will be picked up by the pipeline. Here is a screenshot:
      

   2. FYI, you will lose some performance compared to Triton for most models.

3. Finally, you can combine Triton for inference and ML Flow for the model store. This is the most difficult to setup but would give you the best of both worlds: fast inference and a UI for deploying models at runtime.
   1. We built a ML Flow plugin which supports deploying from ML Flow to Triton here.
   2. Once setup, you should be able to run Morpheus normally and change the models at runtime using the ML Flow UI. Once deployed, Triton will load the new model and start using it for requests.

Let me know if that was helpful.

[gary-sixgen]

3 is good to see! I was unaware of the performance differences but we do already have both deployed after following along with https://github.com/nv-morpheus/Morpheus/blob/branch-23.11/docs/source/cloud_deployment_guide.md

Thank you for sharing these insights.

[raykallen]

Are there other PCAP based models in development? Any good guides for developing our own models?

Our team is not currently developing any other PCAP based models. Is there something you're specifically interested in?

If you want to train a different supervised NLP model that classifies text (like SID) or extracts text found in the payload (like log parsing), we have a few examples of training here: https://github.com/nv-morpheus/Morpheus/tree/branch-23.07/models/training-tuning-scripts With your own labeled dataset, you can use the SID or log parsing training notebooks and scripts to train new models for your use cases.

The pipeline could also be modified to support any non-NLP models you might want to run over PCAPs or just PCAP headers, like tree-based models or the autoencoders for anomaly detection models used in our DFP workflow.