miscalculation of summarized quantize_error

[yoya]

quantize_error summarization has been changed at 6.4.1-3

* magick/quantize.c

- 6.4.1-2

Code: Select all

 node_info->quantize_error+=sqrt((double) (error.red*error.red+
   error.green*error.green+error.blue*error.blue+error.opacity*
   error.opacity));

- 6.4.1-3

Code: Select all

 if (IsColorSimilar(image,p,p+count) == MagickFalse)
   break;
     (omit...)
 node_info->quantize_error+=sqrt((double) (count*error.red*error.red+
   count*error.green*error.green+count*error.blue*error.blue+
   count*error.opacity*error.opacity));

This change causes a color quality degradation of quantized images.

- correct code, I think.

Code: Select all

 if (IsColorSimilar(image,p,p+count) == MagickFalse)
   break;
     (omit...)
 node_info->quantize_error+=count*sqrt((double) (error.red*error.red+
   error.green*error.green+error.blue*error.blue+
   error.opacity*error.opacity));

ref) my patch at 6.x
- https://github.com/gree/YoyaMagick/comm ... e1bf7ac540

[magick]

We can reproduce the problem you posted and added your patch to ImageMagick 6.8.6-5 Beta available by sometime tomorrow. Thanks.

[yoya]

Thank you for your quick response.

At 6.8.6-5, this modified code has wrong calculation.

Code: Select all

node_info->quantize_error+=count*sqrt((double) (error.red*error.red+
  count*error.green*error.green+count*error.blue*error.blue+
  count*error.opacity*error.opacity));

In sqrt all count multiplication must be removed.

Code: Select all

node_info->quantize_error+=count*sqrt((double) (error.red*error.red+
  error.green*error.green+error.blue*error.blue+
  error.opacity*error.opacity));

[magick]

We can reproduce the problem you posted and added your patch to ImageMagick 6.8.6-6 Beta available by sometime tomorrow. Thanks.