Image Compare doesn't work like expected with transparent images

[KiritoStorm]

Magick.NET version

Magick.NET-Q16-AnyCPU-11.3.0

Environment (Operating system, version and so on)

Windows 10 - Version 21H1, 19043.1826

Description

I'm not sure if this is an issue of ImageMagick or the .NET version, but the image compare doesn't work as expected with transparent PNG's.

public bool ImagesEqual(string img1, string img2)
{
    MagickImage first = new MagickImage(img1);
    MagickImage second = new MagickImage(img2);

    double diff = first.Compare(second, new ErrorMetric());
    Console.WriteLine("Images are equal to: " + diff);

    // If the images are equal = NaN, they should at least be 95% equal
    if (double.IsNaN(diff) || diff >= 0.95)
    {
         return true;
    }
    else
    {
         return false;
    }
}

Steps to Reproduce

Put in the method 2 transparent images

Expected result: A value between 0 and 1
Actual result: A value above 1

[dlemstra]

Can you please add the two images so I can see what is happening?

[KiritoStorm]

I used this to make the images transparent (I uploaded both images, every image is a bit different)
image.TransparentChroma(new MagickColor("#1E1E1E"), new MagickColor("#222222"));

[dlemstra]

What is your output? I am getting this as the output: Images are equal to: 0.9908784150008261

[KiritoStorm]

I get Images are equal to: 1,2931903250507701 that's my issue.
Have you compared the transparent ones? Maybe they are compromised thru Github I don't know.

When I compare the orignal images I get: 0,8509939183698039

[KiritoStorm]

I'm still getting values above 1 with Magick.NET 12.0.0

Img1 - Img1 transparent: 1,0522910796486842
Img2 - Img2 transparent: 1,053866595106056
Img1 & Img2 transparent: 1,4993157795479732

[dlemstra]

We tried to make a change inside ImageMagick that turned out to be incorrect so we had to revert that. We are now investigating if this is even wrong. You are now using ErrorMetric.NormalizedCrossCorrelation and you probably want to use ErrorMetric.RootMeanSquared instead.

[urban-warrior]

Thanks for the problem report. We can reproduce it and will have a patch to fix it in the GIT main branch @ https://github.com/ImageMagick/ImageMagick later today. The patch will be available in the beta releases of ImageMagick @ https://imagemagick.org/archive/beta/ by sometime tomorrow.

[KiritoStorm]

Thank you for fixing it 👍
But I don't know how I should test this beta version.
Is there a easy way to test it for me or could you or @dlemstra maybe create a release for it?

[dlemstra]

I will try to add a unit test before the next release to check if this issue was resolved. Already gave it a try on the command line and got this output:

compare -metric ncc img1.png img2.png null:
0.840183

[KiritoStorm]

Thank you a lot 🙏
Is it realistic that I can get a better comparing result when making pixels transparent?
(That's the reason why I'm doing this stuff 😄)

[dlemstra]

I am not sure what you are asking and trying to accomplish. Maybe you could ask for more detailed help in the discussions of the @ImageMagick repository and then start a new discussion here if you need help to translate something to C#.