Post any defects you find in the released or beta versions of the ImageMagick software here. Include the ImageMagick version, OS, and any command-line required to reproduce the problem. Got a patch for a bug? Post it here.
I have a problem when using the compare command with CMYK input images. It seems that the highlight and lowlight colors of the compare result are inverted or converted incorrectly. The red highlight color becomes cyan and the white transparent lowlight color becomes black(?) transparent. Also if I define a highlight color by name i.e. "seagreen" the compare result shows the wrong color as well.
I tested it with two different ImageMagick versions:
The former version is the one we are currently using the latter one is the version we want to upgrade to but has the described issue. You can download my test data using the following link:
What values did you get that tell you they are different? Why do all your outputs have PDF suffixes?
Have you tried comparing in sRGB colorspace? Color inversion is an indication of that problem or possibly the viewer cannot handle it. Is your output image CMYK or RGB?
I have checked with IM 6.8.6.8, 6.9.9.35 and 7.0.7.23. All produce the same metric scores. All images display correctly in Imagemagick display. But the 7.0.7.23 version display inverted when using Mac Preview, whereas the others display properly in Mac Preview.
The PDF suffix is just related to our use case. But the problem also occurs with the other output formats jpg and tif.
Indeed I use macOS (10.12.6) to review the output files but I also use Photoshop to display the compare results and it shows wrong colors too. The output images are CMYK. When comparing RGB images there seems to be no problems regarding the compare result colors.
Also if I take the compare results of both ImageMagick versions and compare these again (using version 6.8.6) every pixel is highlighted red in the output image. So as you also said, there is a difference.