This is the repository of the Met dataset. The official webpage of the dataset can be found here.
The code provides examples for the following:
- How to use the dataset.
- How to evaluate your own method.
- How to reproduce some of the baselines presented in the NeurIPS paper.
In order to run the code you will need:
- Python3
- NumPy
- Faiss library for efficient similarity search (faiss-gpu)
- PyTorch deep learning framework
- The Met dataset (images + ground truth) from the official website
We provide models for descriptor extraction. You can download them here.
We provide pre-extracted descriptors. You can download them here.
Navigate (cd
) to [YOUR_MET_ROOT]/met
. [YOUR_MET_ROOT]
is where you have cloned the github repository.
Descriptor extraction
Example script for extracting descriptors for the images of the Met dataset is located in code/examples/extract_descriptors.py
For detailed explanation of the options run:
python -m code.examples.extract_descriptors -h
kNN classifier & evaluation
Example evaluation script of pre-extracted descriptors with the non-parametric classifier is located in code/examples/knn_eval.py
For detailed explanation of the options run:
python -m code.examples.knn_eval -h
Example (using ground truth and descriptors downloaded from our website, after unzipping both):
python -m code.examples.knn_eval [YOUR_DESCRIPTOR_DIR] --autotune --info_dir [YOUR_GROUND_TRUTH_DIR]
Training with contrastive loss
Example training script for training the embedding model with contrastive loss on the Met training set is located in code/examples/train_contrastive.py
. The trained network can be used for descriptor extraction and kNN classification.
For detailed explanation of the options run:
python -m code.examples.train_contrastive -h
We provide an example for each one of the uses of the provided code. Different combinations should work in the same manner.
Navigate (cd
) to [YOUR_MET_ROOT]/met
. [YOUR_MET_ROOT]
is where you have cloned the github repository.
Descriptor Extraction
Extracting r18SWSL_con-syn+real-closest descriptors using the trained model provided in our website. Ground truth is stored in ./data/ground_truth/
and images are stored in ./data/images
, after both have been extracted from the .zip files. The checkpoint of the model to be loaded is stored in ./data/models/r18SWSL_con-syn+real-closest
. The descriptors will be stored in ./data/descriptors
after the extraction:
python -m code.examples.extract_descriptors ./data/descriptors --info_dir ./data/ground_truth --im_root ./data/ --net r18_contr_loss_gem_fc_swsl --gpuid 0 --netpath ./data/models/r18SWSL_con-syn+real-closest --ms
kNN classifier & evaluation
Evaluating r18SWSL_con-syn+real-closest descriptors. Ground truth is stored in ./data/ground_truth/
and descriptors are stored in ./data/descriptors/r18SWSL_con-syn+real-closest
, after both have been extracted from the .zip files:
python -m code.examples.knn_eval ./data/descriptors/r18SWSL_con-syn+real-closest/ --autotune --info_dir ./data/ground_truth/
Training with contrastive loss
Training a pretrained on ImageNet ResNet18 backbone (FC layer included, initialized with PCAw) using the contrastive loss and the con-syn+real-closest pairs (described in the paper). Ground truth is stored in ./data/ground_truth/
and images are stored in ./data/images
, after both have been extracted from the .zip files. The checkpoints of the model will be stored in ./data/models/r18SWSL_con-syn+real-closest
:
python -m code.examples.train_contrastive --seed 0 --pretrained --pairs_type new_pos+new_neg ./data/models/r18SWSL_con-syn+real-closest --emb_proj --pca --info_dir ./data/ground_truth --im_root ./data/ --gpuid 0
Feel free to give feedback, by sending an email to: [email protected]