Unify sponge gadget

[tessico]

Description

Builds on top of #146. Still WIP, but I'd like to get some comments early on to avoid too much (or rather: even more...) refactoring later.

Refactor native & non-native sponge gadgets:

• add a RescueStateVarGen trait
  • implement for both native and non-native RescueStateVar
  • discuss naming: this could be used outside of the "Rescue" context - it only specifies which fields are native vs. non-native, and how the underlying variables are represented. Perhaps GenericStateVar would be better? Other ideas?
• pull out a unified SpongeGadget trait into mod.rs, generic over RescueStateVarGen
• pull out a unified PermutationGadget trait into mod.rs, generic over RescueStateVarGen

Downside:

• since there are now two implementations of the methods in RescueGadget, the downstream users need to disambiguate the call, e.g. instead of:

self.rescue_sponge_no_padding(&input_labels, 1);

now you need to write

RescueGadget::<RescueStateVar, F, F>::rescue_sponge_no_padding(self, &input_labels, 1);

I feel that the ergonomics of the call is worse.
If you have any idea on how to keep a single SpongeGadget trait instead of RescueGadget/RescueNonNativeGadget, but without requiring disambiguation on every single call, please share :)

On the other hand, we should soon be able to benefit from being generic, for example: generic over a DigestAlgorithmGadget in the implementation of the MerkleTreeGadget, in line with #142:

pub trait DigestAlgorithmGadget<R, T, F>
where
    R: RescueStateVarGen<T, F>,
{
    fn digest(&mut self, data: &[Variable]) -> Variable;
    fn digest_leaf(pos: &usize, elem: &Variable) -> Variable;
}

impl<F: PrimeField + RescueParameter> DigestAlgorithmGadget<RescueStateVar, F, F> {...}

impl<F, D> MerkleTreeGadget<F, F, RescueStateVar, D> for PlonkCircuit<F>
where
    F: PrimeField + RescueParameter,
    D: DigestAlgorithmGadget<RescueStateVar, F, F>,
{
    fn compute_merkle_root(
        &mut self,
        elem: CommittedElemVar,
        path_vars: &MerklePathVar,
    ) -> Result<Variable, CircuitError> {
        let zero_var = self.zero();

        // leaf label = H(0, uid, arc)
        self.digest(&[zero_var, elem.uid, elem.elem]);
      ...

closes: #XXXX

---

Before we can merge this PR, please make sure that all the following items have been
checked off. If any of the checklist items are not applicable, please leave them but
write a little note why.

• Targeted PR against correct branch (main)
• Linked to GitHub issue with discussion and accepted design OR have an explanation in the PR that describes this work.
• Wrote unit tests
• Updated relevant documentation in the code
• Added a relevant changelog entry to the Pending section in CHANGELOG.md
• Re-reviewed Files changed in the GitHub PR explorer

[tessico]

@chancharles92 Yes SpongeStateVar is much better. Done in 726be00.

[chancharles92]

Would it be helpful to have a type-alias or wrapper for the native/non-native gadget's API?

E.g., currently, we need to write: RescueGadget::<RescueStateVar, F, F>::rescue_sponge_no_padding.
It'll be great if the user only needs to write RescueNativeGadget::<F>::rescue_sponge_no_padding.

Similarly, for non-native setting, it'll be great if the user only needs to write RescueNonNativeGadget<F, T>::rescue_sponge_no_padding.

[tessico]

Yes, that would indeed improve the ergonomics of using the gadget. I'll have a look, thx for the suggestion!

[chancharles92]

@tessico Great job! Generally LGTM except for one comment. @mrain can have another round of quick review.

[mrain]

Generally LGTM. Better if Binyi's comment is addressed.

[tessico]

@chancharles92 I've introduced a type alias for the trait.
The downside is that we need to use a dynamic trait now, which as @alxiong has pointed out in some other PR for VRFs, can have a potential performance impact. I haven't ran the benches for this specific case yet though. WDYT?
Also @mrain.

[alxiong]

I've introduced a type alias for the trait. The downside is that we need to use a dynamic trait now,

So trait alias feature is available in rust nightly (see unstable book)
Personally I would stick with the original, more verbose and troublesome way of declaring RescueGadget::<RescueStateVar, F, F>::rescue_sponge_no_padding and add an issue tracking the progress on the stabilization of that rust RFC.

Because after all it's an internal ergonomic issue, and we can tolerate this temporarily imo.

[chancharles92]

I've introduced a type alias for the trait. The downside is that we need to use a dynamic trait now,

So trait alias feature is available in rust nightly (see unstable book) Personally I would stick with the original, more verbose and troublesome way of declaring RescueGadget::<RescueStateVar, F, F>::rescue_sponge_no_padding and add an issue tracking the progress on the stabilization of that rust RFC.

Because after all it's an internal ergonomic issue, and we can tolerate this temporarily imo.

Can we create new traits RescueNativeGadget and RescueNonNativeGadget that inherit corresponding traits? Then we achieve the same as a type alias. (Check here: https://www.worthe-it.co.za/blog/2017-01-15-aliasing-traits-in-rust.html).
We can create an issue and use type alias again after it's stable. The advantage over keeping RescueGadget::<RescueStateVar, F, F>::rescue_sponge_no_padding is that we don't need to change them again everywhere, which is a waste of effort.

[tessico]

can we create new traits RescueNativeGadget and RescueNonNativeGadget that inherit corresponding traits?

No, unfortunately not cleanly, I tried that before as well. I think it's the same issue under the hood, we will run into needing to use the trait as dynamic.

For example, we could define:

// Instead of defining a type alias, define a new supertrait
pub trait RescueNativeGadget<F>: RescueGadget<RescueStateVar, F, F> {}
// blanket impl needed! Otherwise we can't use the methods defined on `PlonkCircuit`
impl<F: RescueParameter> RescueNativeGadget<F> for PlonkCircuit<F> {}

But then it requires you to use it as:

<dyn RescueNativeGadget::<F>>::rescue_permutation(self, state_var)?;

I think the current approach is better in that we explicitly define it as dyn TraitName, and then case use it as if it was just a normal trait.

If we decide to go with current design, we can definitely add a tracking issue in jellyfish for a trait alias.
@chancharles92 @alxiong

[alxiong]

Alright, "keeping the current design" sounds good to me! I'll leave the issue creation to you Marti, thanks!

[chancharles92]

can we create new traits RescueNativeGadget and RescueNonNativeGadget that inherit corresponding traits?

No, unfortunately not cleanly, I tried that before as well. I think it's the same issue under the hood, we will run into needing to use the trait as dynamic.

For example, we could define:

// Instead of defining a type alias, define a new supertrait
pub trait RescueNativeGadget<F>: RescueGadget<RescueStateVar, F, F> {}
// blanket impl needed! Otherwise we can't use the methods defined on `PlonkCircuit`
impl<F: RescueParameter> RescueNativeGadget<F> for PlonkCircuit<F> {}

But then it requires you to use it as:

<dyn RescueNativeGadget::<F>>::rescue_permutation(self, state_var)?;

I think the current approach is better in that we explicitly define it as dyn TraitName, and then case use it as if it was just a normal trait.

If we decide to go with current design, we can definitely add a tracking issue in jellyfish for a trait alias. @chancharles92 @alxiong

I see, sounds good to me. Let's add an issue for that, then we can approve.

[tessico]

Tracking issue here: #165.

[chancharles92]

It looks like we are still using type alias. Are we going with the type alias approach or the original solution as @alxiong suggested? (@alxiong I'm fine with either way, pls take care of the approval.)

[tessico]

I think the current approach is better in that we explicitly define it as dyn TraitName, and then case use it as if it was just a normal trait.

Hmm I thought we were sticking with the type alias. That was the "current solution" at the time of writing :)