Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Check the result of the function in check_determinism #203

Open
tomasol opened this issue Apr 7, 2024 · 1 comment
Open

Check the result of the function in check_determinism #203

tomasol opened this issue Apr 7, 2024 · 1 comment

Comments

@tomasol
Copy link
Contributor

tomasol commented Apr 7, 2024

This is just an idea / feature request, it's totally fine to close the issue if it does not spark any interest.

Currently check_determinism verifies that the randomness state after running the input function twice is the same.
If the input function is truly deterministic, it must return the same result.
Here is how I currently check that a function is deterministic:

#[test]
fn ulid_generation_should_be_deterministic() {
    let mut builder_a = madsim::runtime::Builder::from_env();
    builder_a.check = true;
    let mut builder_b = madsim::runtime::Builder::from_env(); // Builder: Clone would be useful
    builder_b.check = true;
    builder_b.seed = builder_a.seed;
    assert_eq!(
        builder_a.run(|| async { ulid::Ulid::new() }),
        builder_b.run(|| async { ulid::Ulid::new() })
    );
}

It would be nice to have less boilerplate, given that the function returns a value implementing Debug and PartialEq

@wangrunji0408
Copy link
Member

Thanks for sharing your idea!

Besides checking for determinism, we also want to locate the source as soon as possible when uncertainty occurs. Therefore, check_determinism records the entire sequence of random states, and it immediately panics if it finds inconsistencies during the second run. This allows developers to roughly locate the uncertainty source through the backtrace. In comparison, only comparing function return values might miss some cases and doesn't quickly reveal problems. Moreover, this method won't work if the user's function does not return a random number, which I believe is more common in practice.

However, it would be nice to add return value check to the current check_determinism.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

2 participants