Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Harden the pre-tyctxt query system against accidental recomputation #105603

Merged
merged 1 commit into from
Jan 12, 2023

Conversation

oli-obk
Copy link
Contributor

@oli-obk oli-obk commented Dec 12, 2022

While the current compiler has no issues where we take and then compute the query again, in #105462 I accidentally introduced such a case.

I also took the opportunity to remove peek_mut, which is only ever used for global_tcx to then invoke enter. I added an enter method directly on the query.

@rustbot
Copy link
Collaborator

rustbot commented Dec 12, 2022

Thanks for the pull request, and welcome! The Rust team is excited to review your changes, and you should hear from @petrochenkov (or someone else) soon.

Please see the contribution instructions for more information.

@rustbot rustbot added S-waiting-on-review Status: Awaiting review from the assignee but also interested parties. T-compiler Relevant to the compiler team, which will review and decide on the PR/issue. T-rustdoc Relevant to the rustdoc team, which will review and decide on the PR/issue. labels Dec 12, 2022
@rustbot
Copy link
Collaborator

rustbot commented Dec 12, 2022

The Miri subtree was changed

cc @rust-lang/miri

@oli-obk
Copy link
Contributor Author

oli-obk commented Dec 12, 2022

@bors try @rust-timer queue

@rust-timer

This comment has been minimized.

@rustbot rustbot added the S-waiting-on-perf Status: Waiting on a perf run to be completed. label Dec 12, 2022
@bors
Copy link
Contributor

bors commented Dec 12, 2022

⌛ Trying commit d3de0c4 with merge 8148db11639606c756aa0a0fce2b93954d27c19e...

@bors
Copy link
Contributor

bors commented Dec 12, 2022

☀️ Try build successful - checks-actions
Build commit: 8148db11639606c756aa0a0fce2b93954d27c19e (8148db11639606c756aa0a0fce2b93954d27c19e)

@rust-timer

This comment has been minimized.

@rust-timer
Copy link
Collaborator

Finished benchmarking commit (8148db11639606c756aa0a0fce2b93954d27c19e): comparison URL.

Overall result: no relevant changes - no action needed

Benchmarking this pull request likely means that it is perf-sensitive, so we're automatically marking it as not fit for rolling up. While you can manually mark this PR as fit for rollup, we strongly recommend not doing so since this PR may lead to changes in compiler perf.

@bors rollup=never
@rustbot label: -S-waiting-on-perf -perf-regression

Instruction count

This benchmark run did not return any relevant results for this metric.

Max RSS (memory usage)

Results

This is a less reliable metric that may be of interest but was not used to determine the overall result at the top of this comment.

mean range count
Regressions ❌
(primary)
- - 0
Regressions ❌
(secondary)
- - 0
Improvements ✅
(primary)
- - 0
Improvements ✅
(secondary)
-2.4% [-2.4%, -2.4%] 1
All ❌✅ (primary) - - 0

Cycles

This benchmark run did not return any relevant results for this metric.

@rustbot rustbot removed the S-waiting-on-perf Status: Waiting on a perf run to be completed. label Dec 12, 2022
@petrochenkov petrochenkov added S-waiting-on-author Status: This is awaiting some action (such as code changes or more information) from the author. and removed S-waiting-on-review Status: Awaiting review from the assignee but also interested parties. labels Dec 21, 2022
@cjgillot cjgillot assigned cjgillot and unassigned cjgillot Dec 25, 2022
@oli-obk
Copy link
Contributor Author

oli-obk commented Jan 9, 2023

@rustbot ready

@rustbot rustbot added S-waiting-on-review Status: Awaiting review from the assignee but also interested parties. and removed S-waiting-on-author Status: This is awaiting some action (such as code changes or more information) from the author. labels Jan 9, 2023
Copy link
Member

@jyn514 jyn514 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This PR looks like an improvement over the status quo, but Queries is pretty hairy in general ... the whole take() and peek_mut() stuff has never made much sense to me; and why don't we compute the queries on NotComputedYet instead of panicking?

@oli-obk
Copy link
Contributor Author

oli-obk commented Jan 11, 2023

This PR looks like an improvement over the status quo, but Queries is pretty hairy in general ... the whole take() and peek_mut() stuff has never made much sense to me; and why don't we compute the queries on NotComputedYet instead of panicking?

The main issue is that in order to compute the queries we need access to the Queries datastructure, not just the Query itself (in order to be able to call other queries). This requires some fiddling, but then we end up in a situation where the queries could accidentally call each other directly. A bit hard to explain, so maybe look at the necessary change? (90d8197). It's rather verbose at the call sites now (we could avoid that by adding some macros, ... but that will just make it all harder to read).

I am fairly confident that all of this will get simpler with #105462 as most (all?) of these should become real TyCtxt queries

@oli-obk
Copy link
Contributor Author

oli-obk commented Jan 11, 2023

@bors try @rust-timer queue

@rust-timer

This comment has been minimized.

@bors
Copy link
Contributor

bors commented Jan 11, 2023

☀️ Try build successful - checks-actions
Build commit: 74edcc98681086c093ce31fc904515c465558ee3 (74edcc98681086c093ce31fc904515c465558ee3)

@rust-timer

This comment has been minimized.

@rust-timer
Copy link
Collaborator

Finished benchmarking commit (74edcc98681086c093ce31fc904515c465558ee3): comparison URL.

Overall result: ❌ regressions - no action needed

Benchmarking this pull request likely means that it is perf-sensitive, so we're automatically marking it as not fit for rolling up. While you can manually mark this PR as fit for rollup, we strongly recommend not doing so since this PR may lead to changes in compiler perf.

@bors rollup=never
@rustbot label: -S-waiting-on-perf -perf-regression

Instruction count

This is a highly reliable metric that was used to determine the overall result at the top of this comment.

mean range count
Regressions ❌
(primary)
- - 0
Regressions ❌
(secondary)
1.0% [1.0%, 1.1%] 3
Improvements ✅
(primary)
- - 0
Improvements ✅
(secondary)
- - 0
All ❌✅ (primary) - - 0

Max RSS (memory usage)

Results

This is a less reliable metric that may be of interest but was not used to determine the overall result at the top of this comment.

mean range count
Regressions ❌
(primary)
- - 0
Regressions ❌
(secondary)
- - 0
Improvements ✅
(primary)
-3.3% [-3.4%, -3.1%] 2
Improvements ✅
(secondary)
-2.6% [-4.3%, -0.7%] 25
All ❌✅ (primary) -3.3% [-3.4%, -3.1%] 2

Cycles

Results

This is a less reliable metric that may be of interest but was not used to determine the overall result at the top of this comment.

mean range count
Regressions ❌
(primary)
- - 0
Regressions ❌
(secondary)
- - 0
Improvements ✅
(primary)
- - 0
Improvements ✅
(secondary)
-2.6% [-2.6%, -2.6%] 1
All ❌✅ (primary) - - 0

@rustbot rustbot removed the S-waiting-on-perf Status: Waiting on a perf run to be completed. label Jan 11, 2023
@petrochenkov
Copy link
Contributor

r=me after squashing commits.
@rustbot author

@rustbot rustbot added S-waiting-on-author Status: This is awaiting some action (such as code changes or more information) from the author. and removed S-waiting-on-review Status: Awaiting review from the assignee but also interested parties. labels Jan 11, 2023
@oli-obk
Copy link
Contributor Author

oli-obk commented Jan 12, 2023

@bors r=petrochenkov

@bors
Copy link
Contributor

bors commented Jan 12, 2023

📌 Commit 58782a8 has been approved by petrochenkov

It is now in the queue for this repository.

@bors bors added S-waiting-on-bors Status: Waiting on bors to run and complete tests. Bors will change the label on completion. and removed S-waiting-on-author Status: This is awaiting some action (such as code changes or more information) from the author. labels Jan 12, 2023
@bors
Copy link
Contributor

bors commented Jan 12, 2023

⌛ Testing commit 58782a8 with merge 222d1ff...

@bors
Copy link
Contributor

bors commented Jan 12, 2023

☀️ Test successful - checks-actions
Approved by: petrochenkov
Pushing 222d1ff to master...

@bors bors added the merged-by-bors This PR was explicitly merged by bors. label Jan 12, 2023
@bors bors merged commit 222d1ff into rust-lang:master Jan 12, 2023
@rustbot rustbot added this to the 1.68.0 milestone Jan 12, 2023
@oli-obk oli-obk deleted the non_repeatable_queries branch January 12, 2023 17:12
@rust-timer
Copy link
Collaborator

Finished benchmarking commit (222d1ff): comparison URL.

Overall result: no relevant changes - no action needed

@rustbot label: -perf-regression

Instruction count

This benchmark run did not return any relevant results for this metric.

Max RSS (memory usage)

Results

This is a less reliable metric that may be of interest but was not used to determine the overall result at the top of this comment.

mean range count
Regressions ❌
(primary)
5.0% [3.7%, 7.2%] 3
Regressions ❌
(secondary)
2.6% [0.7%, 4.6%] 29
Improvements ✅
(primary)
- - 0
Improvements ✅
(secondary)
- - 0
All ❌✅ (primary) 5.0% [3.7%, 7.2%] 3

Cycles

Results

This is a less reliable metric that may be of interest but was not used to determine the overall result at the top of this comment.

mean range count
Regressions ❌
(primary)
1.0% [1.0%, 1.0%] 1
Regressions ❌
(secondary)
- - 0
Improvements ✅
(primary)
- - 0
Improvements ✅
(secondary)
- - 0
All ❌✅ (primary) 1.0% [1.0%, 1.0%] 1

celinval added a commit to celinval/kani-dev that referenced this pull request Jan 24, 2023
bors added a commit to rust-lang-ci/rust that referenced this pull request Feb 21, 2023
…rochenkov

give the resolver access to TyCtxt

The resolver is now created after TyCtxt is created. Then macro expansion and name resolution are run and the results fed into queries just like before this PR.

Since the resolver had (before this PR) mutable access to the `CStore` and the source span table, these two datastructures are now behind a `RwLock`. To ensure that these are not mutated anymore after the resolver is done, a read lock to them is leaked right after the resolver finishes.

### PRs split out of this one and leading up to it:

* rust-lang#105423
* rust-lang#105357
* rust-lang#105603
* rust-lang#106776
* rust-lang#106810
* rust-lang#106812
* rust-lang#108032
celinval added a commit to model-checking/kani that referenced this pull request Mar 8, 2023
Upgrade our toolchain to `nightly-2023-01-23`. The changes here are related to the following changes:
- rust-lang/rust#104986
- rust-lang/rust#105657
- rust-lang/rust#105603
- rust-lang/rust#105613
- rust-lang/rust#105977
- rust-lang/rust#104645
RalfJung pushed a commit to RalfJung/rust-analyzer that referenced this pull request Apr 20, 2024
give the resolver access to TyCtxt

The resolver is now created after TyCtxt is created. Then macro expansion and name resolution are run and the results fed into queries just like before this PR.

Since the resolver had (before this PR) mutable access to the `CStore` and the source span table, these two datastructures are now behind a `RwLock`. To ensure that these are not mutated anymore after the resolver is done, a read lock to them is leaked right after the resolver finishes.

### PRs split out of this one and leading up to it:

* rust-lang/rust#105423
* rust-lang/rust#105357
* rust-lang/rust#105603
* rust-lang/rust#106776
* rust-lang/rust#106810
* rust-lang/rust#106812
* rust-lang/rust#108032
RalfJung pushed a commit to RalfJung/rust-analyzer that referenced this pull request Apr 27, 2024
give the resolver access to TyCtxt

The resolver is now created after TyCtxt is created. Then macro expansion and name resolution are run and the results fed into queries just like before this PR.

Since the resolver had (before this PR) mutable access to the `CStore` and the source span table, these two datastructures are now behind a `RwLock`. To ensure that these are not mutated anymore after the resolver is done, a read lock to them is leaked right after the resolver finishes.

### PRs split out of this one and leading up to it:

* rust-lang/rust#105423
* rust-lang/rust#105357
* rust-lang/rust#105603
* rust-lang/rust#106776
* rust-lang/rust#106810
* rust-lang/rust#106812
* rust-lang/rust#108032
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
merged-by-bors This PR was explicitly merged by bors. S-waiting-on-bors Status: Waiting on bors to run and complete tests. Bors will change the label on completion. T-compiler Relevant to the compiler team, which will review and decide on the PR/issue. T-rustdoc Relevant to the rustdoc team, which will review and decide on the PR/issue.
Projects
None yet
Development

Successfully merging this pull request may close these issues.

8 participants