Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

💥 Introducing ClusterComplete: Exact SiDB Logic Simulation with well over 50 DBs #390

Open
wants to merge 441 commits into
base: main
Choose a base branch
from

Conversation

wlambooy
Copy link
Contributor

@wlambooy wlambooy commented Mar 7, 2024

Description

This rather extensive PR introduces the groundbreaking ClusterComplete exact simulator to fiction. It relies on the concept of "State Space Pruning in a Cluster Hierarchy", which is demonstrated for the first time with the advent of this simulator as a proof of concept. Jan Drewniok's idea of "Physically Informed Space Pruning" is generalised over charge states, and, above all, the relevant electrostatic potential equations are lifted to higher order to enable reasoning in a cluster hiercharchy. This reasoning allows the Ground State Space algorithm to prune multisets of charge states at any point in the cluster hierarchy where it gets the chance, which is enabled by analysis of potential bounds.

The SiDB cluster hierarchy header file technology/sidb_cluster_hierarchy.hpp is now heavily documented, though it does not have associated tests for all the functions and structures used by Ground State Space. Indirectly, they are well tested, however, since the performance of Ground State Space, and, in turn, ClusterComplete are extremely sensitive to any bugs or inconsistencies in this code. Also, since the methodology of the simulation and pre-simulation are translated directly from a generic theory, no room is left for edge cases, except perhaps the empty layout, and the fact that simulation in base 2 can return 0 physically valid layouts. Both of these have a corresponding test

Checklist:

  • The pull request only contains commits that are related to it.
  • I have added appropriate tests and documentation.
  • I have created/adjusted the Python bindings for any new or updated functionality.
  • I have made sure that all CI jobs on GitHub pass.
  • The pull request introduces no new warnings and follows the project's style guidelines.

@wlambooy
Copy link
Contributor Author

wlambooy commented Mar 7, 2024

The merge brach 'main' in clustercomplete messes with my changed to charge_distribution_surface. This needs to be undone.

Also: the CI has no way of passing at the moment since ALGLIB is not installed. I suppose the following should be added to the workflows:

$ git clone https://github.com/S-Dafarra/alglib-cmake.git
$ cd alglib-cmake
$ mkdir build && cd build
$ cmake ..
$ make
$ [sudo] make install

(clone in the libs folder, the CMakeLists.txt there that was edited in this PR does the rest)

@wlambooy
Copy link
Contributor Author

wlambooy commented Mar 8, 2024

Latest commit reverts a small change (a bug fix in is_ground_state) that apparently make the TTS tests fail. This is unrelated to this PR, so I moved it to Issue #392 .

Copy link

codecov bot commented Mar 8, 2024

Codecov Report

Attention: Patch coverage is 98.63248% with 16 lines in your changes missing coverage. Please review.

Project coverage is 98.48%. Comparing base (8dbfa45) to head (513ca64).
Report is 2 commits behind head on main.

Files Patch % Lines
.../algorithms/simulation/sidb/ground_state_space.cpp 95.33% 9 Missing ⚠️
...st/algorithms/simulation/sidb/time_to_solution.cpp 85.00% 3 Missing ⚠️
...ion/algorithms/simulation/sidb/clustercomplete.hpp 99.35% 1 Missing ⚠️
...lgorithms/simulation/sidb/critical_temperature.hpp 93.33% 1 Missing ⚠️
...orithms/simulation/sidb/sidb_simulation_engine.hpp 91.66% 1 Missing ⚠️
...lude/fiction/technology/sidb_cluster_hierarchy.hpp 99.52% 1 Missing ⚠️
Additional details and impacted files

Impacted file tree graph

@@            Coverage Diff             @@
##             main     #390      +/-   ##
==========================================
+ Coverage   98.22%   98.48%   +0.25%     
==========================================
  Files         216      223       +7     
  Lines       29144    31552    +2408     
  Branches     1391     1436      +45     
==========================================
+ Hits        28626    31073    +2447     
+ Misses        518      479      -39     
Files Coverage Δ
...hms/simulation/sidb/can_positive_charges_occur.hpp 100.00% <100.00%> (ø)
...lation/sidb/exhaustive_ground_state_simulation.hpp 94.44% <100.00%> (ø)
.../algorithms/simulation/sidb/ground_state_space.hpp 100.00% <100.00%> (ø)
...tion/algorithms/simulation/sidb/is_operational.hpp 94.38% <100.00%> (+1.12%) ⬆️
.../algorithms/simulation/sidb/operational_domain.hpp 96.52% <100.00%> (+0.51%) ⬆️
.../fiction/algorithms/simulation/sidb/quickexact.hpp 98.79% <100.00%> (+<0.01%) ⬆️
...de/fiction/algorithms/simulation/sidb/quicksim.hpp 98.59% <100.00%> (-0.02%) ⬇️
...on/algorithms/simulation/sidb/time_to_solution.hpp 97.50% <100.00%> (+0.20%) ⬆️
...fiction/technology/charge_distribution_surface.hpp 99.36% <100.00%> (-0.01%) ⬇️
include/fiction/technology/sidb_charge_state.hpp 95.83% <100.00%> (+4.16%) ⬆️
... and 15 more

... and 145 files with indirect coverage changes


Continue to review full report in Codecov by Sentry.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 1b4c95d...513ca64. Read the comment docs.

@wlambooy
Copy link
Contributor Author

wlambooy commented Mar 8, 2024

It does stop 🥳 Windows CI now sees ALGLIB

@wlambooy
Copy link
Contributor Author

wlambooy commented Mar 8, 2024

Operational Domain fails, inherited from the ❌ on the main branch at the moment

@wlambooy
Copy link
Contributor Author

wlambooy commented Mar 8, 2024

It seems that CodeQL does not find ALGLIB, so at the moment it will inspect empty files where the ALGLIB functionality is.

Also there seems to be something wrong with the linker for the Windows CI wrt ALGLIB. For one Windows CI, any file that indirectly includes sidb_cluster_hierarchy fails.

Copy link
Contributor

@github-actions github-actions bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

clang-tidy made some suggestions

include/fiction/technology/sidb_charge_state.hpp Outdated Show resolved Hide resolved
include/fiction/technology/sidb_charge_state.hpp Outdated Show resolved Hide resolved
include/fiction/technology/sidb_charge_state.hpp Outdated Show resolved Hide resolved
include/fiction/technology/sidb_charge_state.hpp Outdated Show resolved Hide resolved
include/fiction/technology/sidb_charge_state.hpp Outdated Show resolved Hide resolved
include/fiction/technology/sidb_charge_state.hpp Outdated Show resolved Hide resolved
include/fiction/technology/sidb_charge_state.hpp Outdated Show resolved Hide resolved
include/fiction/technology/sidb_charge_state.hpp Outdated Show resolved Hide resolved
include/fiction/technology/sidb_charge_state.hpp Outdated Show resolved Hide resolved
include/fiction/technology/sidb_charge_state.hpp Outdated Show resolved Hide resolved
@marcelwa marcelwa added the enhancement New feature or request label Mar 8, 2024
@marcelwa marcelwa assigned marcelwa and wlambooy and unassigned marcelwa Mar 8, 2024
@wlambooy
Copy link
Contributor Author

wlambooy commented Mar 8, 2024

Thank you Clang-Tidy for the motivation to generalise the charge state iterators. Implemented it with a nice physical intuition:

enum class sidb_state_iter_dir
{
    TO_VALENCE_BAND,
    TO_CONDUCTANCE_BAND
};

Copy link
Contributor

github-actions bot commented Mar 8, 2024

clang-tidy review says "All clean, LGTM! 👍"

2 similar comments
Copy link
Contributor

github-actions bot commented Mar 8, 2024

clang-tidy review says "All clean, LGTM! 👍"

Copy link
Contributor

github-actions bot commented Mar 9, 2024

clang-tidy review says "All clean, LGTM! 👍"

docs/cli.rst Outdated Show resolved Hide resolved
@wlambooy
Copy link
Contributor Author

wlambooy commented Jun 3, 2024

Wow Clang-Tidy started working I see! Any idea what fixed it? I see macOS fails currently because of some unrelated fmt format building issue

@Drewniok
Copy link
Collaborator

Drewniok commented Jun 3, 2024

Wow Clang-Tidy started working I see! Any idea what fixed it? I see macOS fails currently because of some unrelated fmt format building issue

Wow! This is amazing! I don't know why it suddenly works.
Yes, it is not your fault. Marcel is working on fixing it.

@wlambooy
Copy link
Contributor Author

wlambooy commented Jun 3, 2024

Wow Clang-Tidy started working I see! Any idea what fixed it? I see macOS fails currently because of some unrelated fmt format building issue

Copy link
Collaborator

@marcelwa marcelwa left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Dear @wlambooy thank you so much for this wonderful addition to fiction! The code is in a remarkable shape. I have a few comments and suggestions here and there, but nothing too extravagant I hope. One thing I noticed was that compiling the new features still results in some warnings. I would appreciate it if you could address those. After submitting this review, I will commit a few edits I made myself. And then I'll have a deeper look into the CIs.

Comment on lines +1794 to +1815
// Let's figure out what do with this. I think there should be two results, since {0, 1, 1} has a local potential
// of 0.3199995 when all other SiDBs are NEG, which is their only possible charge state. Without error bounds, this
// SiDB should be strictly NEG, but error bounds exist to relax bounds, not make them stricter. Therefore, also the
// NEUT charge state should be accepted since the range is then (0.32 - e, 0.91 + e) = (0.319999, 0.910001).
// (note that I use local potential values according to Vij = - ...)
// SECTION("Test case 4")
// {
// TestType lyt{};
//
// lyt.assign_cell_type({0, 0, 0}, TestType::cell_type::NORMAL);
// lyt.assign_cell_type({0, 1, 1}, TestType::cell_type::NORMAL);
// lyt.assign_cell_type({9, 3, 0}, TestType::cell_type::NORMAL);
// lyt.assign_cell_type({2, 4, 0}, TestType::cell_type::NORMAL);
//
// const clustercomplete_params<cell<TestType>> params{sidb_simulation_parameters{3, -0.32}};
// const quickexact_params<TestType> paramsqe{sidb_simulation_parameters{3, -0.32}};
//
// const auto simulation_results = clustercomplete<TestType>(lyt, params);
// const auto simulation_results2 = quickexact<TestType>(lyt, paramsqe);
//
// CHECK(simulation_results.charge_distributions.size() == 1);
// }
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

What's up with this test case? The comment string indicates there are two possible solutions here. Maybe @Drewniok could also comment on what he thinks is best how do go about it.

Comment on lines +1912 to +1948
// Same as Test case 4.
// SECTION("Test case 7")
// {
// TestType lyt{};
//
// lyt.assign_cell_type({11, 1, 1}, TestType::cell_type::NORMAL);
// lyt.assign_cell_type({15, 1, 1}, TestType::cell_type::NORMAL);
// lyt.assign_cell_type({5, 2, 1}, TestType::cell_type::NORMAL);
// lyt.assign_cell_type({17, 2, 1}, TestType::cell_type::NORMAL);
// lyt.assign_cell_type({20, 3, 0}, TestType::cell_type::NORMAL);
// lyt.assign_cell_type({13, 3, 1}, TestType::cell_type::NORMAL);
// lyt.assign_cell_type({9, 4, 1}, TestType::cell_type::NORMAL);
// lyt.assign_cell_type({2, 5, 1}, TestType::cell_type::NORMAL);
// lyt.assign_cell_type({17, 5, 1}, TestType::cell_type::NORMAL);
// lyt.assign_cell_type({2, 6, 1}, TestType::cell_type::NORMAL);
// lyt.assign_cell_type({9, 6, 1}, TestType::cell_type::NORMAL);
// lyt.assign_cell_type({10, 7, 0}, TestType::cell_type::NORMAL);
// lyt.assign_cell_type({8, 7, 1}, TestType::cell_type::NORMAL);
// lyt.assign_cell_type({17, 9, 0}, TestType::cell_type::NORMAL);
// lyt.assign_cell_type({11, 9, 1}, TestType::cell_type::NORMAL);
// lyt.assign_cell_type({12, 9, 1}, TestType::cell_type::NORMAL);
// lyt.assign_cell_type({5, 10, 0}, TestType::cell_type::NORMAL);
// lyt.assign_cell_type({5, 10, 1}, TestType::cell_type::NORMAL);
// lyt.assign_cell_type({7, 11, 1}, TestType::cell_type::NORMAL);
// lyt.assign_cell_type({13, 11, 1}, TestType::cell_type::NORMAL);
//
// const sidb_simulation_parameters params{2, -0.32};
//
// sidb_simulation_result<TestType> cc_res = clustercomplete(lyt,
// clustercomplete_params<cell<TestType>>{params});
//
// std::sort(cc_res.charge_distributions.begin(), cc_res.charge_distributions.end(),
// [](const auto& lhs, const auto& rhs) { return lhs.get_system_energy() < rhs.get_system_energy();
// });
//
// REQUIRE(cc_res.charge_distributions.size() == 1);
// }
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Can this be removed then?

using ccsim_command = clustercomplete_command;

ALICE_ADD_COMMAND(clustercomplete, "Simulation")
ALICE_ADD_COMMAND(ccsim, "Simulation")
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

If you want to add a shorthand for your command, you can do so in shortcuts.fs and then in your CLI session use < ../shortcuts.fs.

Comment on lines +229 to +231
{"Iteration steps",
std::any_cast<uint64_t>(sim_result_100.additional_simulation_parameters.at("iteration_steps"))},
{"alpha", std::any_cast<double>(sim_result_100.additional_simulation_parameters.at("alpha"))}};
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Aren't these QuickSim stats you're logging here?

Comment on lines +242 to +244
{"Iteration steps",
std::any_cast<uint64_t>(sim_result_111.additional_simulation_parameters.at("iteration_steps"))},
{"alpha", std::any_cast<double>(sim_result_111.additional_simulation_parameters.at("alpha"))}};
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Here too.

Comment on lines +232 to +241
#if (FICTION_ALGLIB_ENABLED)
else if (params.engine == sidb_simulation_engine::CLUSTERCOMPLETE)
{
const clustercomplete_params<cell<Lyt>> cc_params{params.simulation_parameters};

// All physically valid charge configurations are determined for the given layout (`ClusterComplete`
// simulation is used to provide 100 % accuracy for the Critical Temperature).
simulation_results = clustercomplete(layout, cc_params);
}
#endif // FICTION_ALGLIB_ENABLED
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Shall we add an #else case with an assert or something notifying the user that ALGLIB must be installed if ClusterComplete is to be used?

This would then apply to all such cases.

Comment on lines +214 to +227
[[nodiscard]] inline fiction::sidb_simulation_engine get_sim_engine() const noexcept
{
if (sim_engine_str == "ClusterComplete")
{
return fiction::sidb_simulation_engine::CLUSTERCOMPLETE;
}

if (sim_engine_str == "QuickSim")
{
return fiction::sidb_simulation_engine::QUICKSIM;
}

return fiction::sidb_simulation_engine::QUICKEXACT;
}
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

As mentioned above, this function could be replaced with a global one selecting engines.

Comment on lines +276 to +277
LOWER = 0,
UPPER
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Could you add docstrings to the enum values?

Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Wouldn't this test case break down if you set DEBUG_SIDB_CLUSTER_HIERARCHY due to the explicit usage of phmap::flat_hash_set?

Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Please add docstrings to all functions and members (even the private) ones throughout this file. It will greatly help with maintainability in the future.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
enhancement New feature or request
Projects
None yet
Development

Successfully merging this pull request may close these issues.

4 participants