Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Performance improvement when creating the model #3

Merged
merged 7 commits into from
Apr 11, 2023

Conversation

GregoryWullimann
Copy link
Collaborator

@GregoryWullimann GregoryWullimann commented Mar 28, 2023

  • Refactored some part of the model creation. Mostly reduced the number of iteration over the player dataset.
  • Added a wrapper function to log the execution time, useful for keeping track of slow part methods. Logging can be enabled/disabled changing LOG_RUNTIME in input.py
  • Added requirements.txt so it's easier to install dependencies
  • Converted XSLS to CSV to improve data reading performance

Model creation now takes about 10 seconds to create. create_basic_constraints is the slowest function now and can be definitely improved.

Processing time create_var: 0.11 seconds
Processing time create_basic_constraints: 7.37 seconds
Processing time create_max_club_constraint: 0.29 seconds
Processing time create_unique_league_constraint: 0.11 seconds
Processing time create_min_country_constraint: 0.22 seconds
Processing time create_rarity_2_constraint: 0.03 seconds
Processing time create_chemistry_constraint: 3.38 seconds

@Regista6
Copy link
Owner

Regista6 commented Mar 30, 2023

Extended your code. Here's what I got:

Processing time create_var: 1.06 seconds
Processing time create_basic_constraints: 0.33 seconds
Processing time create_max_club_constraint: 0.05 seconds
Processing time create_unique_league_constraint: 0.08 seconds
Processing time create_min_country_constraint: 0.08 seconds
Processing time create_rarity_2_constraint: 0.02 seconds
Processing time create_chemistry_constraint: 2.89 seconds

Need to do some testing with different inputs. Thanks very much!

@Regista6 Regista6 added the enhancement New feature or request label Mar 30, 2023
@Regista6
Copy link
Owner

Regista6 commented Apr 8, 2023

@Regista6 Regista6 force-pushed the model-creation-performance-and-minor-fixes branch from 2bd43a1 to 607a3c4 Compare April 9, 2023 08:10
@Regista6 Regista6 merged commit 32ae79e into master Apr 11, 2023
@Regista6 Regista6 mentioned this pull request Apr 20, 2023
@Regista6 Regista6 deleted the model-creation-performance-and-minor-fixes branch April 22, 2023 18:16
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
enhancement New feature or request
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants