This release does not encompass major changes to the repository, but it is associated with snapshotted weights available on Globus.
The Network saved here achieves about 0.8 mIoU on the validation set. Weights available here.
(As a note, the data is available here )
To run inference with this model, do something like this:
python /home/cadams/ThetaGPU/CosmicTagger/bin/exec.py --config-name SCC_21 mode=inference \
mode.weights_location=output/tensorflow/scc21/untether_sota_8nodes/checkpoints/ \
run.id=test-live \
data.data_directory=/lus/eagle/projects/datascience/CosmicTaggerSCC_21/data_files/ \
data.file=cosmic_tagging_val.h5 \
framework=tensorflow run.iterations=100Inference results look like this (8xA100, mixed precision with XLA enabled):
2022-04-20 14:04:33,103 - INFO - Inference report: 218 batches processed for 6976 entries.
2022-04-20 14:04:33,103 - INFO - plane0/Total_Accuracy: 0.9997
2022-04-20 14:04:33,104 - INFO - plane0/Non_Bkg_Accuracy: 0.9404
2022-04-20 14:04:33,104 - INFO - plane0/Neutrino_IoU: 0.6720
2022-04-20 14:04:33,104 - INFO - plane0/Cosmic_IoU: 0.9220
2022-04-20 14:04:33,104 - INFO - plane1/Total_Accuracy: 0.9997
2022-04-20 14:04:33,104 - INFO - plane1/Non_Bkg_Accuracy: 0.9491
2022-04-20 14:04:33,105 - INFO - plane1/Neutrino_IoU: 0.6779
2022-04-20 14:04:33,105 - INFO - plane1/Cosmic_IoU: 0.9259
2022-04-20 14:04:33,105 - INFO - plane2/Total_Accuracy: 0.9997
2022-04-20 14:04:33,105 - INFO - plane2/Non_Bkg_Accuracy: 0.9467
2022-04-20 14:04:33,105 - INFO - plane2/Neutrino_IoU: 0.6534
2022-04-20 14:04:33,105 - INFO - plane2/Cosmic_IoU: 0.9234
2022-04-20 14:04:33,105 - INFO - Average/Total_Accuracy: 0.9997
2022-04-20 14:04:33,106 - INFO - Average/Non_Bkg_Accuracy: 0.9454
2022-04-20 14:04:33,106 - INFO - Average/Neutrino_IoU: 0.6678
2022-04-20 14:04:33,106 - INFO - Average/Cosmic_IoU: 0.9238
2022-04-20 14:04:33,106 - INFO - Average/mIoU: 0.7958
2022-04-20 14:04:33,106 - INFO - Total time to batch_process: 115.9773
2022-04-20 14:04:33,106 - INFO - Total time to batch process except first iteration: 115.1700, throughput: 60.2935
2022-04-20 14:04:33,106 - INFO - Total time to batch process except first two iterations: 114.6399, throughput: 60.2932
2022-04-20 14:04:33,106 - INFO - Total time to batch process last 40 iterations: 21.1240, throughput: 60.5947