Chronos is a system for the automation of the entire evaluation workflow including the set-up of the evaluation, its execution, and the subsequent analysis of the results. It allows to evaluate a wide range of applications from simple databases to complex polystores or from plain text retrieval to complex multimedia search engines and thus fosters the reproducibility of evaluations.
Systems evaluations are an important part of development and empirical research in computer science. They encompass the systematic assessment of the run-time characteristics of systems based on one or several parameters. Considering all possible parameter settings is often a very tedious and time-consuming task with many manual activities.
- MySQL server database
- PHP 7.4 or higher
- Apache2 with active mod_rewrite
- Apache config
AllowOverride Allneeds to be set - Git
Copy the file install.php into the webroot folder of the server and call it in the browser.
Follow the instructions there to install the system and setup Chronos.
After a successful setup you can log in with admin as username and password as password. Before you start with your first evaluations, adjust the settings in the admin interface according to your environment and your requirements.
To perform your first evaluation, please follow the following steps:
- Integrate the Chronos Agent in your existing evaluation client.
- Register your system in Chronos Control (Systems -> Add system). Please notice that the VCS settings are optional.
- Configure the parameters of your evaluation client (Systems -> [your system] -> Configure Parameters).
- Configure how the results of a single evaluation job should be visualized (Systems -> [your system] -> Configure Job Results).
- Similar to this, configure how the results of an complete evaluation (consisting of multiple jobs) should be visualized (Systems -> [your system] -> Configure Overall Results).
- Get the system id (Systems -> [your system] -> System ID) and add it to your evaluation client.
- Start your client and specify an environment.
- In Chronos Control, create a project (Projects -> Add Project) for your evaluation campaign.
- Create an experiment (Projects -> [your project] -> Create Experiment) within your project.
- Trigger the first evaluation (Projects -> [your project] -> [your experiment] -> Run evaluation).
See the open issues for a list of proposed features (and known issues).
We highly welcome contributions to the Chronos project. If you would like to contribute, please fork the repository and submit your changes as a pull request.
The Chronos user interface shines thanks to the beautiful AdminLTE template.
The MIT License (MIT)