Use your own model

[Nazim0001]

Hi,
I would like to use my own model with WSInfer. But I don't know how I need to do, and what kind of information I need to add in JSON file?
Could you please help me?
Best
Nazim

[kaczmarj]

hi @Nazim0001 - you can use one of the JSON files in the model zoo as a starting point, and replace things specific to your model. i have pasted the JSON for the pancancer lymphocyte classification model.

please note that the architecture key is unused but because i included it in the json schema, you need to include it. the value isn't used, so fill that in with anything you want.

you can validate your JSON against our schema using the following command:

wsinfer-zoo validate-config config.json

once you have the JSON file, you will also need to have your model in torchscript format. when you have your model and the config JSON, run the model using the following command:

wsinfer run --wsi-dir slides/ --results-dir results/ --model-path path/to/torchscript.pt --config config.json

here is an example config JSON:

{
  "spec_version": "1.0",
  "architecture": "inception_v4nobn",
  "num_classes": 2,
  "class_names": [
    "Other",
    "Lymphocytes"
  ],
  "patch_size_pixels": 100,
  "spacing_um_px": 0.5,
  "transform": [
    {
      "name": "Resize",
      "arguments": {
        "size": 299
      }
    },
    {
      "name": "ToTensor"
    },
    {
      "name": "Normalize",
      "arguments": {
        "mean": [
          0.5,
          0.5,
          0.5
        ],
        "std": [
          0.5,
          0.5,
          0.5
        ]
      }
    }
  ]
}

[Nazim0001]

Thank you Jakub for your reply. I thought that it would be possible to use WSInfer for a new task like to quantify a staining, right?

…

On Sun, Mar 17, 2024, 13:49 Jakub Kaczmarzyk, ***@***.***> wrote: hi @Nazim0001 <https://github.com/Nazim0001> - you can use one of the JSON files in the model zoo as a starting point, and replace things specific to your model. i have pasted the JSON for the pancancer lymphocyte classification model <https://huggingface.co/kaczmarj/pancancer-lymphocytes-inceptionv4.tcga/blob/main/config.json> . please note that the architecture key is unused but because i included it in the json schema, you need to include it. the value isn't used, so fill that in with anything you want. you can validate your JSON against our schema using the following command: wsinfer-zoo validate-config config.json once you have the JSON file, you will also need to have your model in torchscript format. when you have your model and the config JSON, run the model using the following command: wsinfer run --wsi-dir slides/ --results-dir results/ --model-path path/to/torchscript.pt --config config.json here is an example config JSON: { "spec_version": "1.0", "architecture": "inception_v4nobn", "num_classes": 2, "class_names": [ "Other", "Lymphocytes" ], "patch_size_pixels": 100, "spacing_um_px": 0.5, "transform": [ { "name": "Resize", "arguments": { "size": 299 } }, { "name": "ToTensor" }, { "name": "Normalize", "arguments": { "mean": [ 0.5, 0.5, 0.5 ], "std": [ 0.5, 0.5, 0.5 ] } } ] } — Reply to this email directly, view it on GitHub <#221 (comment)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/AY5TW4BTITEWPI4KG5ZUEW3YYWGMVAVCNFSM6AAAAABEZ7YTXKVHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMZDAMBSGQ2TCNZYHE> . You are receiving this because you were mentioned.Message ID: ***@***.***>

[kaczmarj]

Could you describe your use case in more detail?Wsinfer does patch classification.Best,JakubOn Mar 17, 2024, at 10:39, Nazim0001 ***@***.***> wrote: Thank you Jakub for your reply. I thought that it would be possible to use WSInfer for a new task like to quantify a staining, right?

On Sun, Mar 17, 2024, 13:49 Jakub Kaczmarzyk, ***@***.***> wrote: hi @Nazim0001 <https://github.com/Nazim0001> - you can use one of the JSON files in the model zoo as a starting point, and replace things specific to your model. i have pasted the JSON for the pancancer lymphocyte classification model <https://huggingface.co/kaczmarj/pancancer-lymphocytes-inceptionv4.tcga/blob/main/config.json> . please note that the architecture key is unused but because i included it in the json schema, you need to include it. the value isn't used, so fill that in with anything you want. you can validate your JSON against our schema using the following command: wsinfer-zoo validate-config config.json once you have the JSON file, you will also need to have your model in torchscript format. when you have your model and the config JSON, run the model using the following command: wsinfer run --wsi-dir slides/ --results-dir results/ --model-path path/to/torchscript.pt --config config.json here is an example config JSON: { "spec_version": "1.0", "architecture": "inception_v4nobn", "num_classes": 2, "class_names": [ "Other", "Lymphocytes" ], "patch_size_pixels": 100, "spacing_um_px": 0.5, "transform": [ { "name": "Resize", "arguments": { "size": 299 } }, { "name": "ToTensor" }, { "name": "Normalize", "arguments": { "mean": [ 0.5, 0.5, 0.5 ], "std": [ 0.5, 0.5, 0.5 ] } } ] } — Reply to this email directly, view it on GitHub <#221 (comment)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/AY5TW4BTITEWPI4KG5ZUEW3YYWGMVAVCNFSM6AAAAABEZ7YTXKVHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMZDAMBSGQ2TCNZYHE> . You are receiving this because you were mentioned.Message ID: ***@***.***>

—Reply to this email directly, view it on GitHub, or unsubscribe.You are receiving this because you commented.Message ID: ***@***.***>

[Nazim0001]

I would like to use wsinfer to count the number of cells with staining of pdl1 over the number of total cells. What do you think? Is it possible ? On Sun, Mar 17, 2024, 15:54 Jakub Kaczmarzyk, ***@***.***> wrote:

…

Could you describe your use case in more detail?Wsinfer does patch classification.Best,JakubOn Mar 17, 2024, at 10:39, Nazim0001 ***@***.***> wrote: Thank you Jakub for your reply. I thought that it would be possible to use WSInfer for a new task like to quantify a staining, right? On Sun, Mar 17, 2024, 13:49 Jakub Kaczmarzyk, ***@***.***> wrote: > hi @Nazim0001 <https://github.com/Nazim0001> - you can use one of the > JSON files in the model zoo as a starting point, and replace things > specific to your model. i have pasted the JSON for the pancancer > lymphocyte classification model > < https://huggingface.co/kaczmarj/pancancer-lymphocytes-inceptionv4.tcga/blob/main/config.json> > . > > please note that the architecture key is unused but because i included it > in the json schema, you need to include it. the value isn't used, so fill > that in with anything you want. > > you can validate your JSON against our schema using the following command: > > wsinfer-zoo validate-config config.json > > once you have the JSON file, you will also need to have your model in > torchscript format. when you have your model and the config JSON, run the > model using the following command: > > wsinfer run --wsi-dir slides/ --results-dir results/ --model-path path/to/torchscript.pt --config config.json > > here is an example config JSON: > > { > "spec_version": "1.0", > "architecture": "inception_v4nobn", > "num_classes": 2, > "class_names": [ > "Other", > "Lymphocytes" > ], > "patch_size_pixels": 100, > "spacing_um_px": 0.5, > "transform": [ > { > "name": "Resize", > "arguments": { > "size": 299 > } > }, > { > "name": "ToTensor" > }, > { > "name": "Normalize", > "arguments": { > "mean": [ > 0.5, > 0.5, > 0.5 > ], > "std": [ > 0.5, > 0.5, > 0.5 > ] > } > } > ] > } > > — > Reply to this email directly, view it on GitHub > <#221 (comment)>, > or unsubscribe > < https://github.com/notifications/unsubscribe-auth/AY5TW4BTITEWPI4KG5ZUEW3YYWGMVAVCNFSM6AAAAABEZ7YTXKVHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMZDAMBSGQ2TCNZYHE> > . > You are receiving this because you were mentioned.Message ID: > ***@***.***> > —Reply to this email directly, view it on GitHub, or unsubscribe.You are receiving this because you commented.Message ID: ***@***.***> — Reply to this email directly, view it on GitHub <#221 (comment)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/AY5TW4FXZMMWQCZSPZLUINDYYWVCDAVCNFSM6AAAAABEZ7YTXKVHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMZDAMBSGQ4TQNBQGI> . You are receiving this because you were mentioned.Message ID: ***@***.***>

[kaczmarj]

if you have a patch classification model that can do this, then yes, you could use wsinfer to run that on slides. the wsinfer model zoo does not include any models that count cells, so you would have to provide your own model.

but counting cells is not usually a patch classification task. you might want to consider other methods.

[ajr82]

Hi Jakub @kaczmarj
I have used Wsinfer (built-in models) on QuPath and it is great. We would like to train and use custom models.

Would pytorch models trained using Yolov8 be compatible with Wsinfer? These models have a .pt format. They can also be converted into Torchscript models. https://docs.ultralytics.com/modes/export/#arguments

[kaczmarj]

hi @ajr82 - if the models do multi-class classification or multi-label classification, then yes, they would be compatible. the YOLOv8 classification models might work after being converted to torchscript. wsinfer expects that the forward pass outputs a tensor of logits per patch.

from my experience with yolo models, some might also return a bounding boxes or segmentations. in that case, the wsinfer code would have to be modified.

do you only need classification? or are you trying to perform other tasks too?

[ajr82]

Thank you for getting back!
Yes, Yolo models either return segmentations or bounding boxes. We were hoping to use these models to annotate specific structures in renal biopsies (like glomeruli, tubules) and eventually do morphometric analyzes.