Profile and Optimize Mining Code

[gregorycoppola]

#3012 discusses how blocks often have few tx's in them.

This is a result of two things:

1. the timeout times to stop processing
2. the speed of tx processing

I have measured processing speed on my GCP box as

INFO [1643037789.034557] [src/chainstate/stacks/miner.rs:1934] [main] num_considered 269 seconds_elapsed 183.046 fraction 0.6804684014869888 

This is about 1.47 transactions per second. This means that, to put 1k tx's in a block, it would take 680s=11minutes. This means that it will be hard to put 1k tx's in most blocks.

Maybe we can improve processing speed, to drastically fill more blocks. Eg. we can pre-cache the MARF look-ups that will be needed for a given set of mempool tx's before mining begins.

[gregorycoppola]

Here is a profile I made.

Looks like query_row_and_then takes up most of the time, but not sure if the graph is showing the relationships right, in that it looks like these calls are outside of process_transaction, but that might not be true.

[jcnelson]

Closing in favor of your other issues related to improving the miner performance