PoX-4 Signing Keys Stacking-State #4092

setzeus · 2023-11-25T22:29:50Z

PoX-4 Signing Key Stacking-State Update

Summary of Changes

This PR is meant to introduce, not complete, the core logic around signing keys in PoX-4 & address #4057 . The focus is specifically on updating the 'stacking-state' map & all the functions that read & write from it. This includes updates to the following public functions:

(stack-stx)
(delegate-stack-stx)
(stack-extend)
(delegate-stack-extend)

what this does not include
Getters & setters to the new map 'reward-cycle-signing-keys,' that'll be in another PR. This PR also does not include any logic that'll be used in relevant issues #4058, #4059, & #3970.

Testing

How were these changes tested?

This was only tested locally for syntax errors / clarinet check.

What future testing should occur?

As soon as this is merged I'll open an issue with the user stories that require testing for @moodmosaic to work into pox_4_tests.rs.

Checklist:

My code follows the style guidelines of this project
I have performed a self-review of my code
I have commented my code, particularly in hard-to-understand areas
I have made corresponding changes to the documentation
My changes generate no new warnings
I have added tests that prove my fix is effective or that my feature works
New and existing unit tests pass locally with my changes
Any dependent changes have been merged and published in downstream modules

codecov · 2023-11-25T22:59:25Z

Codecov Report

Attention: 24 lines in your changes are missing coverage. Please review.

Comparison is base (8cffc38) 85.08% compared to head (166ac8f) 64.10%.
Report is 13 commits behind head on next.

Files	Patch %	Lines
...kslib/src/chainstate/stacks/boot/contract_tests.rs	0.00%	24 Missing ⚠️

Additional details and impacted files

@@             Coverage Diff             @@
##             next    #4092       +/-   ##
===========================================
- Coverage   85.08%   64.10%   -20.99%     
===========================================
  Files         429      429               
  Lines      302009   302143      +134     
===========================================
- Hits       256976   193700    -63276     
- Misses      45033   108443    +63410

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

setzeus · 2023-11-28T00:44:25Z

Additional context re: using 'delegate-stack-stx' vs. 'stack-aggregation-commit-index' to submit a pool's signing-key for delegates that registered without a signing-key here:

#4082

setzeus · 2023-11-28T13:55:20Z

Note: per Clarity WG morning discussion it's been noted that our preferred workflow here is to actively work on a version of PoX-4 that exists in /contrib per #4094 & then periodically copy that over to the more hardened PoX-4 that'll live in /boot.

Once #4094 is merged in I'll make these adjust only to the PoX-4 version in /contrib for now.

(unless there's reasonable push back on this workflow)

stackslib/src/chainstate/stacks/boot/pox-4.clar

kantai · 2023-11-29T20:24:19Z

stackslib/src/chainstate/stacks/boot/pox-4.clar

-        delegated-to: (optional principal)
+        delegated-to: (optional principal),
+        ;; signing key for Nakamoto+, only 'none' when delegated & before delegate calls 'stack-aggregation-commit-indexed
+        signing-key: (optional (buff 33))


It seems like this could just be (buff 33) rather than an (optional (buff 33)): the delegate sets this value in delegate-stack-stx, which is also the first time the stacker-state is instantiated for the delegator.

Does it make sense to allow delegators enforce a signing key? If yes, should it be stored in a separate map?

Does it make sense to allow delegators enforce a signing key? If yes, should it be stored in a separate map?

Why would the delegator call delegate-stack-stx if the user supplied the wrong key?

Sorry, I meant the user/pool member. My question does not make sense here.

I agree, that stack-aggregation-commit can't change the signing key.

stackslib/src/chainstate/stacks/boot/pox-4.clar

kantai

Echoing Jude's comments, and just had a few more

kantai · 2023-11-30T15:26:11Z

@setzeus -- this patch adds a unit test which would check that the contract instantiates:

diff --git a/stackslib/src/chainstate/stacks/boot/contract_tests.rs b/stackslib/src/chainstate/stacks/boot/contract_tests.rs
index bd7941f73..d8450f0e7 100644
--- a/stackslib/src/chainstate/stacks/boot/contract_tests.rs
+++ b/stackslib/src/chainstate/stacks/boot/contract_tests.rs
@@ -60,6 +60,8 @@ lazy_static! {
     pub static ref POX_CONTRACT_TESTNET: QualifiedContractIdentifier = boot_code_id("pox", false);
     pub static ref POX_2_CONTRACT_TESTNET: QualifiedContractIdentifier =
         boot_code_id("pox-2", false);
+    pub static ref POX_4_CONTRACT_TESTNET: QualifiedContractIdentifier =
+        boot_code_id("pox-4", false);
     pub static ref COST_VOTING_CONTRACT_TESTNET: QualifiedContractIdentifier =
         boot_code_id("cost-voting", false);
     pub static ref USER_KEYS: Vec<StacksPrivateKey> =
@@ -559,6 +561,31 @@ impl HeadersDB for TestSimHeadersDB {
     }
 }
 
+#[test]
+fn pox_4_instantiates() {
+    let mut sim = ClarityTestSim::new();
+    sim.epoch_bounds = vec![0, 1, 2];
+    let delegator = StacksPrivateKey::new();
+
+    let expected_unlock_height = POX_TESTNET_CYCLE_LENGTH * 4;
+
+    // execute past 2.1 epoch initialization
+    sim.execute_next_block(|_env| {});
+    sim.execute_next_block(|_env| {});
+    sim.execute_next_block(|_env| {});
+
+    sim.execute_next_block(|env| {
+        env.initialize_versioned_contract(
+            POX_4_CONTRACT_TESTNET.clone(),
+            ClarityVersion::Clarity2,
+            &boot::POX_4_TESTNET_CODE,
+            None,
+            ASTRules::PrecheckSize,
+        )
+        .unwrap()
+    });
+}
+
 #[test]
 fn pox_2_contract_caller_units() {
     let mut sim = ClarityTestSim::new();

The test passes with the most recent changes:

$ cargo test pox_4_instantiates
...
running 1 test
test chainstate::stacks::boot::contract_tests::pox_4_instantiates ... ok

test result: ok. 1 passed; 0 failed; 0 ignored; 0 measured; 1105 filtered out; finished in 0.04s
...

It also (correctly) fails if applied to the commit 6771f81.

friedger

I try to understand whether the same logic for the pox-address should be applied for the signing key for pools: If the pool operator decides to use a different signer key, the pool members rewards are not really at risk. However, would pool members like to decide whether the pool operator can delegate to one or the other signer? I think yes, therefore delegate-stx should also contain a signing key in the delegation-state

friedger · 2023-12-01T14:31:15Z

@setzeus There at least one rust functions that create stack-stx transactions with 4 instead of 5 arguments that makes the unit tests fail.
I found one in blocks.rs: https://github.com/stacks-network/stacks-blockchain/blob/cef8f87c59d326b5d30d9f2e6baf258d820da6b1/stackslib/src/chainstate/stacks/db/blocks.rs#L4222

from the logs of the test:

WARN [1701440351.250362] [stackslib/src/chainstate/stacks/db/transactions.rs:1045] [chainstate::nakamoto::coordinator::tests::test_simple_nakamoto_coordinator_10_tenures_10_blocks] Contract-call encountered an analysis error at runtime, contract_name: ST000000000000000000002AMW42H.pox-4, function_name: stack-stx, function_args: [u1000000000000000000, (tuple (hashbytes 0x253029fcdb225da88ef8434ed2bd32ff353983cf) (version 0x00)), u34, u12], error: IncorrectArgumentCount(5, 4)

setzeus · 2023-12-01T14:46:14Z

Ahhh okay, thx @friedger, so updating PoX-4 stacking parameters broke existing tests somewhere else* I follow. Looking into it.

stackslib/src/chainstate/stacks/boot/pox-4.clar

muneeb-ali · 2023-12-06T19:17:17Z

Great to see the changes approved here! 🚀

jcnelson

Thank you for updating the stacking interfaces with these new methods! Before merging, I must insist that you add unit tests for them. It's not enough to see that PoX-4 instantiates; you must also verify that the public keys get added to the stacking state. Thanks!

jcnelson · 2023-12-06T20:35:35Z

stackslib/src/chainstate/nakamoto/coordinator/tests.rs

@@ -283,8 +283,9 @@ fn test_simple_nakamoto_coordinator_bootup() {
 #[test]
 fn test_simple_nakamoto_coordinator_1_tenure_10_blocks() {
    let mut test_signers = TestSigners::default();
-    let mut peer = boot_nakamoto(function_name!(), vec![], test_signers.aggregate_public_key);
-    let private_key = peer.config.private_key.clone();
+    //let mut peer = boot_nakamoto(function_name!(), vec![], test_signers.aggregate_public_key);


Please remove dead code

jcnelson · 2023-12-06T20:35:51Z

stackslib/src/chainstate/nakamoto/coordinator/tests.rs

-    let private_key = peer.config.private_key.clone();
+    //let mut peer = boot_nakamoto(function_name!(), vec![], test_signers.aggregate_public_key);
+    //let private_key = peer.config.private_key.clone();
+    let private_key = StacksPrivateKey::new();


Why is this a new key? Can you continue to please use the same key so we get consistent output?

Recommendation from @kantai to pass/debug those unit tests, reverting back to commented out above I get in both places where I switched out peer.config for a new private key:

let private_key = peer.config.private_key.clone(); | ^^^^ not found in this scope

The problem is because you've commented out the code that instantiates peer

stackslib/src/chainstate/stacks/boot/contract_tests.rs

jcnelson · 2023-12-07T16:41:19Z

An update from the testing WG. This PR is blocked on getting the pox_4.rs tests in place, which is being handled in #4106. Once this is in place, it should be merged into this PR, so then it will become possible to test that the signing keys were indeed set.

jcnelson · 2023-12-13T20:48:40Z

stackslib/src/chainstate/stacks/boot/mod.rs

-        //                           (pox-addr (tuple (version (buff 1)) (hashbytes (buff 32))))
-        //                           (burn-height uint)
-        //                           (lock-period uint))
+        let public_key = StacksPublicKey::from_private(key);


This key is the key used to sign the transaction, not the DKG signing key. There needs to be a separate argument to make_pox_4_lockup for that signing key.

jcnelson · 2023-12-13T20:49:23Z

stackslib/src/chainstate/stacks/boot/pox-4.clar

@@ -117,7 +118,9 @@
        ;;  previous stack-stx calls, or prior to an extend)
        reward-set-indexes: (list 12 uint),
        ;; principal of the delegate, if stacker has delegated
-        delegated-to: (optional principal)
+        delegated-to: (optional principal),
+        ;; signing key for Nakamoto, only 'none' when delegated & before delegate calls stack-aggregation-commit-indexed


~~I don't think this is ever none if we're setting the signing key on delegate-stack-stx (which IIRC is the plan now)~~

EDIT: this comment is out of date

jcnelson · 2023-12-13T20:51:27Z

stackslib/src/chainstate/stacks/boot/pox-4.clar

@@ -591,7 +605,8 @@
 (define-public (stack-stx (amount-ustx uint)
                          (pox-addr (tuple (version (buff 1)) (hashbytes (buff 32))))
                          (start-burn-ht uint)
-                          (lock-period uint))
+                          (lock-period uint)
+                          (signing-key (buff 33)))


Can you add a comment about signing-key to the method doc?

Updated a sentence & added the following to comment doc above function:

;; ...Stacker is responsible for providing a valid signing key. ... ;; You need to provide a signing-key used to allocate signature-weight to a valid signer

jcnelson · 2023-12-13T20:52:02Z

stackslib/src/chainstate/stacks/boot/pox-4.clar

@@ -829,7 +845,8 @@
                                   (amount-ustx uint)
                                   (pox-addr { version: (buff 1), hashbytes: (buff 32) })
                                   (start-burn-ht uint)
-                                   (lock-period uint))
+                                   (lock-period uint)


Same here -- can you add a comment to the method doc about the signing key?

Updated a sentence & added the following to comment doc above function:

;; * You need to provide a signing-key used to allocate signature-weight to a valid signer

jcnelson · 2023-12-13T20:54:12Z

testnet/stacks-node/src/mockamoto.rs

@@ -824,6 +824,9 @@ impl MockamotoNode {
            Some(AddressHashMode::SerializeP2PKH),
        );

+        let public_key = StacksPublicKey::from_private(&self.miner_key);


I think there instead needs to be a separate signing_key field in MockamotoNode. Same for the Nakamoto node. Perhaps it can be part of the Keychain?

moodmosaic

It would be nice with some tests to ensure everything works as intended:

signing-key in stacking-state:
- Verify that the signing-key is correctly set.
Modified Functions (stack-stx, delegate-stack-stx, etc.):
- Ensure all functions modified to include the signing-key handle it correctly.
stack-extend, delegate-stack-extend:
- Test handling of updated-signing-key, ensuring proper updates in the stacking state.
get-signer-info-for-cycle:
- Verify retrieval of correct signer info for various inputs.

@jcnelson, @setzeus, could you share your thoughts on these test cases or suggest additional ones that might be relevant?

moodmosaic · 2023-12-14T16:13:15Z

stackslib/src/chainstate/stacks/boot/pox-4.clar

@@ -27,6 +27,7 @@
 (define-constant ERR_DELEGATION_WRONG_REWARD_SLOT 29)
 (define-constant ERR_STACKING_IS_DELEGATED 30)
 (define-constant ERR_STACKING_NOT_DELEGATED 31)
+(define-constant ERR_STACK_EXTEND_NO_SIGNING_KEY 32)


Where is this constant used?

Unused, removed.

jcnelson · 2023-12-19T16:10:11Z

@moodmosaic Yes, please test all of those in pox_4_tests.rs.

jcnelson · 2023-12-19T16:11:09Z

contrib/core-contract-tests/tests/bns/name_register_test.ts

@@ -98,7 +98,7 @@ Clarinet.test({
                ], 
                cases[0].nameOwner.address),
        ]);
-        assertEquals(block.height, 2);
+        assertEquals(block.height, 3);


This looks like there was an off-by-one error somewhere, with all these changes. Can you elaborate more on why all of these block heights are now +1?

The pox-4 is in a different epoch than bns, therefore booting needs one more block

jcnelson · 2023-12-19T16:12:23Z

stackslib/src/chainstate/stacks/boot/mod.rs

@@ -1798,7 +1794,7 @@ pub mod test {
            boot_code_test_addr(),
            POX_4_NAME,
            "stack-extend",
-            vec![Value::UInt(lock_period), addr_tuple],
+            vec![Value::UInt(lock_period), addr_tuple, Value::Optional(OptionalData {data: None})],


Don't pass None; instead, make this an argument in Rust which gets passed to stack-extend.

jcnelson · 2023-12-19T16:12:42Z

stackslib/src/chainstate/stacks/boot/pox-4.clar

-(define-constant ADDRESS_VERSION_NATIVE_P2WPKH 0x04)
-(define-constant ADDRESS_VERSION_NATIVE_P2WSH 0x05)
-(define-constant ADDRESS_VERSION_NATIVE_P2TR 0x06)
+;; (define-constant ADDRESS_VERSION_NATIVE_P2WPKH 0x04)


Please don't commit commented-out code

jcnelson · 2023-12-19T16:13:34Z

stackslib/src/chainstate/stacks/boot/pox-4.clar


 ;; PoX disabling threshold (a percent)
 (define-constant POX_REJECTION_FRACTION u25)

 ;; Valid values for burnchain address versions.
-;; These first four correspond to address hash modes in Stacks 2.1,
-;; and are defined in pox-mainnet.clar and pox-testnet.clar (so they
-;; cannot be defined here again).
 ;; (define-constant ADDRESS_VERSION_P2PKH 0x00)


If you're now using is-in-mainnet instead of a separate pox-4-mainnet.clar, then where are these constants defined now?

jcnelson · 2023-12-19T16:14:29Z

stackslib/src/chainstate/stacks/boot/pox-4.clar

@@ -508,6 +535,14 @@
    (and (>= lock-period MIN_POX_REWARD_CYCLES)
         (<= lock-period MAX_POX_REWARD_CYCLES)))

+;; Is the given burn block height in the prepare phase?
+;; This computes `((height - first-burnchain-block-height) + pox-prepare-cycle-length) % pox-reward-cycle-length) < pox-prepare-cycle-length`.
+(define-read-only (check-prepare-phase (height uint))


@setzeus This needs to be in #4164

jcnelson · 2023-12-19T16:15:28Z

stackslib/src/chainstate/stacks/boot/pox-4.clar

@@ -545,6 +580,10 @@
    (asserts! (check-pox-lock-period num-cycles)
              (err ERR_STACKING_INVALID_LOCK_PERIOD))

+    ;; stacking must not happen during prepare phase
+    (asserts! (not (check-prepare-phase burn-block-height))


Please revert this -- it has not been decided yet that stacking will be disabled in the prepare phase

Yes. Reverting all the work here to un-do rebasing from #4094.

moodmosaic · 2023-12-21T13:16:31Z

stackslib/src/chainstate/stacks/boot/pox-4.clar

 ;; Once the delegate has stacked > minimum, the delegate should call stack-aggregation-commit
 (define-public (delegate-stack-stx (stacker principal)
                                   (amount-ustx uint)
                                   (pox-addr { version: (buff 1), hashbytes: (buff 32) })
                                   (start-burn-ht uint)
-                                   (lock-period uint))
+                                   (lock-period uint)
+                                   (signing-key (buff 33)))


@jcnelson, we'd like to get this method (delegate-stack-stx) under test, with @setzeus. In this diff, there's the signing-key key that's added.

Looking at PoX-3 tests, some candidates to copy over to PoX-4 are

delegate_stack_increase

delegate_extend_pox_3

pox_3_getters

stack_aggregation_increase

All of these tests seem to call into delegate-stack-stx. The question is:

Are all those tests relevant in the context of PoX-4

Which one would be the best to start with, so it can be added into this PR.

setzeus · 2023-12-22T14:36:54Z

Work picked up by @MarvinJanssen in #4209 to add required tests in addition to base tests - changing to draft until #4209 is merged in then will close.

setzeus · 2024-01-03T14:52:52Z

Closing since #4209 is now merged in.

setzeus self-assigned this Nov 28, 2023

setzeus requested review from jcnelson, kantai and friedger November 28, 2023 00:41

setzeus requested a review from MarvinJanssen November 28, 2023 13:12

jcnelson reviewed Nov 28, 2023

View reviewed changes

stackslib/src/chainstate/stacks/boot/pox-4.clar Outdated Show resolved Hide resolved

jcnelson reviewed Nov 28, 2023

View reviewed changes

stackslib/src/chainstate/stacks/boot/pox-4.clar Outdated Show resolved Hide resolved

jcnelson reviewed Nov 28, 2023

View reviewed changes

stackslib/src/chainstate/stacks/boot/pox-4.clar Outdated Show resolved Hide resolved

jcnelson reviewed Nov 28, 2023

View reviewed changes

stackslib/src/chainstate/stacks/boot/pox-4.clar Outdated Show resolved Hide resolved

jcnelson reviewed Nov 28, 2023

View reviewed changes

stackslib/src/chainstate/stacks/boot/pox-4.clar Outdated Show resolved Hide resolved

jcnelson reviewed Nov 28, 2023

View reviewed changes

stackslib/src/chainstate/stacks/boot/pox-4.clar Outdated Show resolved Hide resolved

jcnelson reviewed Nov 28, 2023

View reviewed changes

stackslib/src/chainstate/stacks/boot/pox-4.clar Outdated Show resolved Hide resolved

kantai reviewed Nov 29, 2023

View reviewed changes

stackslib/src/chainstate/stacks/boot/pox-4.clar Outdated Show resolved Hide resolved

kantai reviewed Nov 29, 2023

View reviewed changes

friedger requested changes Dec 1, 2023

View reviewed changes

friedger reviewed Dec 3, 2023

View reviewed changes

stackslib/src/chainstate/stacks/boot/pox-4.clar Outdated Show resolved Hide resolved

setzeus requested a review from friedger December 4, 2023 21:03

kantai approved these changes Dec 4, 2023

View reviewed changes

friedger approved these changes Dec 5, 2023

View reviewed changes

setzeus mentioned this pull request Dec 6, 2023

Tests in develop/next are not behaving the same as master #4128

Closed

jcnelson requested changes Dec 6, 2023

View reviewed changes

jcnelson reviewed Dec 13, 2023

View reviewed changes

setzeus added the argon3.0 label Dec 14, 2023

setzeus added this to the Nakamoto Milestone 0.3 (code name: Argon) milestone Dec 14, 2023

moodmosaic reviewed Dec 14, 2023

View reviewed changes

setzeus force-pushed the feat/pox-4-signing-keys branch from 214d72d to 81273b6 Compare December 19, 2023 02:54

setzeus mentioned this pull request Dec 19, 2023

Next/PoX-4 Register Signing Keys #4190

Closed

8 tasks

jcnelson reviewed Dec 19, 2023

View reviewed changes

Undo rebase from 4094

72ca439

moodmosaic force-pushed the feat/pox-4-signing-keys branch from 81273b6 to 72ca439 Compare December 19, 2023 16:35

setzeus added 4 commits December 20, 2023 07:18

continue undoing #4094 rebase

1e2e8e0

updated existing pox-4 tests to work with updated stacking funcs

c66946a

removed finished todo

013266a

format overlook

7352f3e

moodmosaic reviewed Dec 21, 2023

View reviewed changes

removed map for tighter PR focus

166ac8f

setzeus mentioned this pull request Dec 22, 2023

PoX-4 signer key read/write with base Rust tests #4209

Merged

5 tasks

setzeus marked this pull request as draft December 22, 2023 14:37

setzeus closed this Jan 3, 2024

PoX-4 Signing Keys Stacking-State #4092

PoX-4 Signing Keys Stacking-State #4092

Conversation

setzeus commented Nov 25, 2023 • edited Loading

PoX-4 Signing Key Stacking-State Update

Summary of Changes

Testing

How were these changes tested?

What future testing should occur?

Checklist:

codecov bot commented Nov 25, 2023 • edited Loading

Codecov Report

setzeus commented Nov 28, 2023

setzeus commented Nov 28, 2023

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

kantai left a comment

Choose a reason for hiding this comment

kantai commented Nov 30, 2023

friedger left a comment

Choose a reason for hiding this comment

friedger commented Dec 1, 2023 • edited Loading

setzeus commented Dec 1, 2023

muneeb-ali commented Dec 6, 2023

jcnelson left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

setzeus Dec 7, 2023 • edited Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

jcnelson commented Dec 7, 2023

Choose a reason for hiding this comment

jcnelson Dec 13, 2023 • edited Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

setzeus Dec 14, 2023 • edited Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

moodmosaic left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

jcnelson commented Dec 19, 2023

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

moodmosaic Dec 21, 2023 • edited Loading

Choose a reason for hiding this comment

setzeus commented Dec 22, 2023

setzeus commented Jan 3, 2024

setzeus commented Nov 25, 2023 •

edited

Loading

codecov bot commented Nov 25, 2023 •

edited

Loading

friedger commented Dec 1, 2023 •

edited

Loading

setzeus Dec 7, 2023 •

edited

Loading

jcnelson Dec 13, 2023 •

edited

Loading

setzeus Dec 14, 2023 •

edited

Loading

moodmosaic Dec 21, 2023 •

edited

Loading