dvyukov jobs optimizations #4957
Open
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Collaborator
dvyukov
commented
Jul 1, 2024
•
dvyukov
commented
- prog: reduce amount of hint replacements
- pkg/fuzzer: optimize smash jobs
- pkg/fuzzer: remove signal rotation
- pkg/fuzzer: try to triage on different VMs
- pkg/fuzzer: tune corpus triage
- pkg/fuzzer: issue minimize attempts in parallel
- executor: restart procs more deterministically
- pkg/fuzzer: deflake hints 3 times instead of 2
dvyukov
force-pushed
the
dvyukov-jobs-optimizations
branch
from
070e676
to
d76aed1
Compare
dvyukov
force-pushed
the
dvyukov-jobs-optimizations
branch
2 times, most recently
from
d3b04af
to
972f3fe
Compare
a-nogikh
reviewed
Jul 3, 2024
dvyukov
force-pushed
the
dvyukov-jobs-optimizations
branch
from
972f3fe
to
53b7406
Compare
Distribute triage requests to different VMs.
Use 2/6 criteria during corpus triage. See the large added comment for details.
Hints can produce insane amounts of candidates (up to 30K). So run deflaking one more time, even if it reduces amount of candidates by few percents, it's still profitable.
Currently we force restart in rpcserver, but this has 2 problems: 1. It does not know the proc where the requets will land. 2. It does not take into account if the proc has already restarted recently for other reasons. Restart procs in executor only if they haven't restarted recenlty. Also make it deterministic. Given all other randomess we have, there does not seem to be a reason to use randomized restarts and restart after fewer/more runs. Also restart only after corpus triage. Corpus triage is slow already and there does not seem to be enough benefit to restart during corpus triage. Also restart at most 1 proc at a time, since there are lots of serial work in the kernel.
We are getting too many generated candidates, the fuzzer may not keep up with them at all (hints jobs keep growing infinitely). If a hint indeed came from the input w/o transformation, then we should guess it on the first attempt (or at least after few attempts). If it did not come from the input, or came with a non-trivial transformation, then any number of attempts won't help. So limit the total number of attempts (until the next restart).
dvyukov
force-pushed
the
dvyukov-jobs-optimizations
branch
from
53b7406
to
cfda6fe
Compare
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.