Improve Engine API abstraction

[mattsse]

Describe the feature

Currently we provide a way to abstract parts of the engine API (PayloadAttributes) because these are different eth vs OP

reth/crates/node-api/src/engine/mod.rs

Lines 19 to 24 in a4c0349

	/// The types that are used by the engine API.
	pub trait EngineTypes:
	serde::de::DeserializeOwned + fmt::Debug + Unpin + Send + Sync + Clone
	{
	/// The RPC payload attributes type the CL node emits via the engine API.
	type PayloadAttributes: PayloadAttributes + Unpin;

with alloy-rs/alloy#227 we now have the case where the return type: ExecutionPayload is also no longer consistent which means those trait conversion functions are no longer entirely consistent:

reth/crates/node-api/src/engine/traits.rs

Lines 15 to 32 in a4c0349

	/// Represents a built payload type that contains a built [SealedBlock] and can be converted into
	/// engine API execution payloads.
	pub trait BuiltPayload: Send + Sync + std::fmt::Debug {
	/// Returns the built block (sealed)
	fn block(&self) -> &SealedBlock;
	/// Returns the fees collected for the built block
	fn fees(&self) -> U256;
	/// Converts the type into the response expected by `engine_getPayloadV1`
	fn into_v1_payload(self) -> ExecutionPayloadV1;
	/// Converts the type into the response expected by `engine_getPayloadV2`
	fn into_v2_payload(self) -> ExecutionPayloadEnvelopeV2;
	/// Converts the type into the response expected by `engine_getPayloadV2`
	fn into_v3_payload(self) -> ExecutionPayloadEnvelopeV3;
	}

The way it currently works is, the EngineTypes configure

• The PayloadAttributes type: type we get from the CL
• BuiltPayload: the type that payload builder service builds, which currently must be convertible into the versioned execution payload envelope

I think we now also the ExecutionPayloadV* types to be configurable, so that we can have clean types for all networks and don't need additional Options for OP for example, see alloy PR

TODO

we can solve this in a number of ways, but I think we want an additional associated type for the ExecutionPayload on the EngineTypes trait and then a requirement that the configured BuiltPayload can be converted into this type.

perhaps we want an associated type for each version or an enum, I assume 3 types might be easier to handle from engine impl POV:

reth/crates/rpc/rpc-engine-api/src/engine_api.rs

Lines 253 to 256 in a4c0349

	pub async fn get_payload_v3(
	&self,
	payload_id: PayloadId,
	) -> EngineApiResult<ExecutionPayloadEnvelopeV3> {

This would also require to introduce OP specific V3 type on alloy-rpc

@fgimenez would you be interested in this?

Additional context

No response

[fgimenez]

@fgimenez would you be interested in this?

absolutely! thx a lot @mattsse will start with it asap

[mattsse]

fyi @fgimenez this is also relevant here

#6331

We should do TryFrom or similar