Choice: Meta-Learning - Using prior data to warmstart optimization
This is our team's approach at finding a combination of search algorithm and multi-fidelity scheduler that performs well on the dataset deepweeds
.
The search algorithm is a Bayesian Optimizer that employs a random forest regressor as a surrogate model. This surrogate model is warmstarted on objective function evaluations that were supplied.
The scheduler employs a prediction and grace period strategy to create unique adaptive fidelity behavior.
Download the Repository
git clone https://github.com/MarcSpeckmann/Meta-Learning-Using-Prior-Data-to-Warmstart-Optimization.git
First you need to install miniconda.
Make sure there is no environment that is already named "automl"
Subsequently, run these commands:
make install
Activate the environment
conda activate automl
You may want to adjust the settings for how many concurrent trials to run and how many CPUs and GPUs to use per trial. These settings are found starting at line 281 in main.py
Our test system was able to handle up to about 12 concurrent trials, with fractional resource allocations.
Run the main experiment file
python main.py
For reproducing the experiments detailed on the poster run the specific experiment_<searcher>_<scheduler>_<1,2,3>.py
The experiments were carried out on a computer with the following resources:
- AMD EPYC 7543 32-Core Processor (But only using 4 cores, 4vCPU)
- NVIDIA A100-SXM4-40GB
- 528 GB RAM
The test set accuracy will automatically be displayed after finishing the set runtime.