Autoquant Config

[HDCharles]

Summary: this adds support for an autoquant config

#creating a AutoQuantConfig
aq_config = AutoQuantConfig(model)

#quantizing a model using hte AutoQuantConfig
aq_config.apply_to_model(model)

#save/load an AutoQuantConfig
aq_config.save(file_path)
aq_config.load(file_path) or AutoQuantConfig(file_path)

Test Plan:

python test_integration.py -k "test_autoquant_config"

Reviewers:

Subscribers:

Tasks:

Tags:

[pytorch-bot]

🔗 Helpful Links

🧪 See artifacts and rendered test results at hud.pytorch.org/pr/pytorch/ao/368

• 📄 Preview Python docs built from this PR

Note: Links to docs will display an error until the docs builds have been completed.

✅ No Failures

As of commit fa0ebfb with merge base bc2f8b7 ():
💚 Looks good so far! There are no failures yet. 💚

This comment was automatically generated by Dr. CI and updates every 15 minutes.

[msaroufim]

sorry I keep harping on names :P

But this sounds like it's a Cache instead?

In which case wouldn't something like this be clearer torch.autoquant(model, cache_path = "test.pkl") which would call apply_to_model() on behalf of the user?

And shouldn't you always save because most users would want shorter quantization times unless they're debugging cache issues?

[HDCharles]

The cache (https://github.com/pytorch/ao/blob/main/torchao/quantization/autoquant.py#L22) is something different. I still need to write automatic serialization for that.

unless you're saying this new thing is reall a Cache? How would you define the difference between a cache and a config?

[msaroufim]

I think of configs as a class representing a common set of input arguments to a function while a cache is something that saves the result of some search process