Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[Framework] Add dependencies check for code object freezing #14064

Merged
merged 2 commits into from
Jul 22, 2024
Merged

Conversation

movekevin
Copy link
Contributor

Description

Type of Change

  • New feature
  • [ x ] Bug fix
  • Breaking change
  • Performance improvement
  • Refactoring
  • Dependency update
  • Documentation update
  • Tests

Which Components or Systems Does This Change Impact?

  • Validator Node
  • Full Node (API, Indexer, etc.)
  • Move/Aptos Virtual Machine
  • [ x ] Aptos Framework
  • Aptos CLI/SDK
  • Developer Infrastructure
  • Other (specify)

How Has This Been Tested?

E2E tests

Key Areas to Review

Checklist

  • I have read and followed the CONTRIBUTING doc
  • I have performed a self-review of my own code
  • I have commented my code, particularly in hard-to-understand areas
  • I identified and added all stakeholders and component owners affected by this change as reviewers
  • I tested both happy and unhappy path of the functionality
  • I have made corresponding changes to the documentation

Copy link

trunk-io bot commented Jul 19, 2024

⏱️ 3h 42m total CI duration on this PR
Job Cumulative Duration Recent Runs
execution-performance / single-node-performance 25m 🟩
forge-framework-upgrade-test / forge 16m 🟩
forge-e2e-test / forge 16m 🟩
rust-move-tests 15m 🟩
rust-move-tests 15m 🟩
rust-move-tests 14m 🟩
rust-move-unit-coverage 14m 🟩
rust-move-unit-coverage 13m 🟩
forge-compat-test / forge 13m 🟩
rust-move-tests 10m 🟩
rust-move-unit-coverage 9m 🟩
check-dynamic-deps 9m 🟩🟩🟩🟩🟩 (+2 more)
test-target-determinator 8m 🟩
general-lints 8m 🟩🟩🟩🟩 (+1 more)
execution-performance / test-target-determinator 8m 🟩
rust-cargo-deny 7m 🟩🟩🟩🟩 (+1 more)
cli-e2e-tests / run-cli-tests 7m 🟩
rust-doc-tests 6m 🟩
check 4m 🟩
semgrep/ci 3m 🟩🟩🟩🟩🟩 (+2 more)
file_change_determinator 1m 🟩🟩🟩🟩🟩 (+2 more)
file_change_determinator 1m 🟩🟩🟩🟩 (+2 more)
permission-check 24s 🟩🟩🟩🟩🟩 (+2 more)
permission-check 22s 🟩🟩🟩🟩🟩 (+2 more)
permission-check 16s 🟩🟩🟩🟩🟩 (+2 more)
permission-check 15s 🟩🟩🟩🟩🟩 (+2 more)
file_change_determinator 14s 🟩
rust-move-unit-coverage 7s
Backport PR 5s 🟥
rust-move-tests 4s
rust-move-unit-coverage 4s
permission-check 4s 🟩
rust-move-tests 4s
permission-check 2s 🟩
determine-docker-build-metadata 2s 🟩
rust-move-unit-coverage 1s
rust-move-tests 1s

🚨 3 jobs on the last run were significantly faster/slower than expected

Job Duration vs 7d avg Delta
test-target-determinator 8m 5m +63%
execution-performance / single-node-performance 25m 15m +60%
execution-performance / test-target-determinator 8m 5m +52%

settingsfeedbackdocs ⋅ learn more about trunk.io

@movekevin movekevin force-pushed the freeze-fix branch 3 times, most recently from 3850c4f to cc9b120 Compare July 19, 2024 23:08
Copy link

codecov bot commented Jul 19, 2024

Codecov Report

All modified and coverable lines are covered by tests ✅

Project coverage is 58.9%. Comparing base (0f49e1c) to head (c154a34).
Report is 22 commits behind head on main.

❗ There is a different number of reports uploaded between BASE (0f49e1c) and HEAD (c154a34). Click for more details.

HEAD has 1 upload less than BASE
Flag BASE (0f49e1c) HEAD (c154a34)
2 1
Additional details and impacted files
@@             Coverage Diff             @@
##             main   #14064       +/-   ##
===========================================
- Coverage    71.5%    58.9%    -12.6%     
===========================================
  Files        2330      824     -1506     
  Lines      460902   198033   -262869     
===========================================
- Hits       329781   116770   -213011     
+ Misses     131121    81263    -49858     

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

// This would increase the amount of gas used, but this is a rare operation and it's rare to have many packages
// in a single code object.
vector::for_each(registry.packages, |pack| {
check_dependencies(code_object_addr, &pack);
Copy link
Contributor

@junkil-park junkil-park Jul 20, 2024

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This check_dependencies function returns allowed_module_deps that are normally used in the native bytecode dep verification:

/// is passed on to the native layer to verify that bytecode dependencies are actually what is pretended here.

Is it fine to ignore the return value?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yeah here in this freeze function we only care about visibility checks as other checks should have already been performed when this package was first deployed or last updated

@JohnChangUK
Copy link
Member

JohnChangUK commented Jul 21, 2024

LGTM, maybe one more test to verify freezing an upgradeable/mutable package with immutable dependencies should work too.

Edit: Added this in!

@davidiw davidiw enabled auto-merge (squash) July 22, 2024 14:50

This comment has been minimized.

This comment has been minimized.

This comment has been minimized.

Copy link
Contributor

✅ Forge suite compat success on 1c2ee7082d6eff8c811ee25d6f5a7d00860a75d5 ==> c154a346e7c65baa9019cdaf57e5a0b27a46cca3

Compatibility test results for 1c2ee7082d6eff8c811ee25d6f5a7d00860a75d5 ==> c154a346e7c65baa9019cdaf57e5a0b27a46cca3 (PR)
1. Check liveness of validators at old version: 1c2ee7082d6eff8c811ee25d6f5a7d00860a75d5
compatibility::simple-validator-upgrade::liveness-check : committed: 9888.741094532335 txn/s, latency: 3485.8833483134185 ms, (p50: 2700 ms, p90: 6900 ms, p99: 17200 ms), latency samples: 378280
2. Upgrading first Validator to new version: c154a346e7c65baa9019cdaf57e5a0b27a46cca3
compatibility::simple-validator-upgrade::single-validator-upgrading : committed: 5815.785909429077 txn/s, latency: 4694.913249911253 ms, (p50: 4000 ms, p90: 6700 ms, p99: 7300 ms), latency samples: 112680
compatibility::simple-validator-upgrade::single-validator-upgrade : committed: 6444.488103210587 txn/s, latency: 4703.056964069438 ms, (p50: 4600 ms, p90: 5400 ms, p99: 8200 ms), latency samples: 247700
3. Upgrading rest of first batch to new version: c154a346e7c65baa9019cdaf57e5a0b27a46cca3
compatibility::simple-validator-upgrade::half-validator-upgrading : committed: 7200.950300020357 txn/s, latency: 3715.6232256203116 ms, (p50: 4000 ms, p90: 4200 ms, p99: 4300 ms), latency samples: 138640
compatibility::simple-validator-upgrade::half-validator-upgrade : committed: 6531.638990437131 txn/s, latency: 4841.616250489237 ms, (p50: 4600 ms, p90: 7100 ms, p99: 9100 ms), latency samples: 255500
4. upgrading second batch to new version: c154a346e7c65baa9019cdaf57e5a0b27a46cca3
compatibility::simple-validator-upgrade::rest-validator-upgrading : committed: 11451.543310792089 txn/s, latency: 2339.744072469476 ms, (p50: 2400 ms, p90: 2900 ms, p99: 3100 ms), latency samples: 203120
compatibility::simple-validator-upgrade::rest-validator-upgrade : committed: 10863.814363745521 txn/s, latency: 3034.1744014836463 ms, (p50: 2900 ms, p90: 4600 ms, p99: 6000 ms), latency samples: 355880
5. check swarm health
Compatibility test for 1c2ee7082d6eff8c811ee25d6f5a7d00860a75d5 ==> c154a346e7c65baa9019cdaf57e5a0b27a46cca3 passed
Test Ok

Copy link
Contributor

✅ Forge suite realistic_env_max_load success on c154a346e7c65baa9019cdaf57e5a0b27a46cca3

two traffics test: inner traffic : committed: 9302.913291709478 txn/s, submitted: 9539.456119539342 txn/s, failed submission: 0.05260013961082136 txn/s, expired: 236.54282782986368 txn/s, latency: 2781.714588009793 ms, (p50: 2700 ms, p90: 3400 ms, p99: 4200 ms), latency samples: 3537220
two traffics test : committed: 99.91865734843222 txn/s, latency: 2207.5815 ms, (p50: 2000 ms, p90: 2300 ms, p99: 8100 ms), latency samples: 2000
Latency breakdown for phase 0: ["QsBatchToPos: max: 0.236, avg: 0.216", "QsPosToProposal: max: 1.200, avg: 0.616", "ConsensusProposalToOrdered: max: 0.326, avg: 0.293", "ConsensusOrderedToCommit: max: 0.404, avg: 0.387", "ConsensusProposalToCommit: max: 0.693, avg: 0.681"]
Max round gap was 1 [limit 4] at version 1054381. Max no progress secs was 5.84632 [limit 15] at version 3883654.
Test Ok

Copy link
Contributor

✅ Forge suite framework_upgrade success on 1c2ee7082d6eff8c811ee25d6f5a7d00860a75d5 ==> c154a346e7c65baa9019cdaf57e5a0b27a46cca3

Compatibility test results for 1c2ee7082d6eff8c811ee25d6f5a7d00860a75d5 ==> c154a346e7c65baa9019cdaf57e5a0b27a46cca3 (PR)
Upgrade the nodes to version: c154a346e7c65baa9019cdaf57e5a0b27a46cca3
framework_upgrade::framework-upgrade::full-framework-upgrade : committed: 1066.771706091283 txn/s, submitted: 1068.7950876202233 txn/s, failed submission: 2.023381528940263 txn/s, expired: 2.023381528940263 txn/s, latency: 2927.948503688093 ms, (p50: 2100 ms, p90: 5200 ms, p99: 11100 ms), latency samples: 94900
framework_upgrade::framework-upgrade::full-framework-upgrade : committed: 1108.4687489332177 txn/s, submitted: 1109.8642194522272 txn/s, failed submission: 1.395470519009506 txn/s, expired: 1.395470519009506 txn/s, latency: 2858.7942299622323 ms, (p50: 2100 ms, p90: 5100 ms, p99: 8400 ms), latency samples: 95320
5. check swarm health
Compatibility test for 1c2ee7082d6eff8c811ee25d6f5a7d00860a75d5 ==> c154a346e7c65baa9019cdaf57e5a0b27a46cca3 passed
Upgrade the remaining nodes to version: c154a346e7c65baa9019cdaf57e5a0b27a46cca3
framework_upgrade::framework-upgrade::full-framework-upgrade : committed: 1060.4832357965254 txn/s, submitted: 1062.4708137061816 txn/s, failed submission: 1.9875779096561286 txn/s, expired: 1.9875779096561286 txn/s, latency: 3124.899104539775 ms, (p50: 2100 ms, p90: 6000 ms, p99: 10900 ms), latency samples: 96040
Test Ok

@davidiw davidiw merged commit 97780dc into main Jul 22, 2024
102 of 105 checks passed
@davidiw davidiw deleted the freeze-fix branch July 22, 2024 17:39
junkil-park pushed a commit that referenced this pull request Jul 22, 2024
* Refactor object deployment CLI commands

* Refactor object deployment CLI commands

---------

Co-authored-by: JohnChangUK <[email protected]>
movekevin added a commit that referenced this pull request Jul 22, 2024
…14079)

* Refactor object deployment CLI commands

* Refactor object deployment CLI commands

---------

Co-authored-by: Kevin <[email protected]>
Co-authored-by: JohnChangUK <[email protected]>
junkil-park pushed a commit that referenced this pull request Jul 23, 2024
* Refactor object deployment CLI commands

* Refactor object deployment CLI commands

---------

Co-authored-by: JohnChangUK <[email protected]>
fEst1ck pushed a commit to fEst1ck/aptos-core that referenced this pull request Jul 23, 2024
…bs#14064)

* Refactor object deployment CLI commands

* Refactor object deployment CLI commands

---------

Co-authored-by: JohnChangUK <[email protected]>
sherry-x pushed a commit that referenced this pull request Jul 23, 2024
* Refactor object deployment CLI commands

* Refactor object deployment CLI commands

---------

Co-authored-by: JohnChangUK <[email protected]>
sherry-x pushed a commit that referenced this pull request Jul 23, 2024
* Refactor object deployment CLI commands

* Refactor object deployment CLI commands

---------

Co-authored-by: JohnChangUK <[email protected]>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

4 participants