By Rongmei Lin, Weiyang Liu, Zhen Liu, Chen Feng, Zhiding Yu, James Rehg, Li Xiong, Le Song
CoMHE is released under the MIT License (refer to the LICENSE file for details).
Inspired by the Thomson problem in physics where the distribution of multiple propelling electrons on a unit sphere can be modeled via minimizing some potential energy, hyperspherical energy minimization has demonstrated its potential in regularizing neural networks and improving their generalization power. See our previous work -- MHE for an in-depth introduction.
Here we propose the compressive minimum hyperspherical energy (CoMHE) as a more effective regularization for neural networks (compared to the original MHE). Specifically, CoMHE utilizes projection mappings to reduce the dimensionality of neurons and minimizes their hyperspherical energy. According to different designs for the projection mapping, we consider several well-performing variants. Welcome to try our CoMHE in your work!
Our CoMHE is accepted to CVPR 2020 and the full paper is available on arXiv and here.
If you find our work useful in your research, please consider to cite:
@InProceedings{Lin20CoMHE,
title={Regularizing Neural Networks via Minimizing Hyperspherical Energy},
author={Lin, Rongmei and Liu, Weiyang and Liu, Zhen and Feng, Chen and Yu, Zhiding
and Rehg, James M. and Xiong, Li and Song, Le},
booktitle={CVPR},
year={2020}
}
We also provide a short video introduction to help interested readers quickly go over our work and understand the essence of CoMHE. Please click the following figure to watch the Youtube video.
Python 3.6
TensorFlow 1.14.0
git clone https://github.com/rmlin/CoMHE.git
wget https://www.cs.toronto.edu/~kriz/cifar-100-python.tar.gz
# run random projection CoMHE
cd random_projection
python train.py
# run angle-preserving projection CoMHE
cd angle_projection
python train.py
# run adversarial projection CoMHE
cd adversarial_projection
python train.py
If you want to change the hyperparameter settings of CoMHE (e.g., Random projection CoMHE), please refer to the file train.py for different input arguments such as dimension and number of projections.
The expected training and testing dynamics (loss and accuracy) can be found in the corresponding log folder.