Issues with -clamp and -depth in HDR

[pipe]

I stumbled upon a problem with -clamp and -depth in the HDRI version of imagemagick, I'm not sure if I've understood the -clamp operation here, but I'll show the problem:

My original image is a scanned 16-bit TIFF image, resized to something manageable to show the problem: hdr_depth_clamp_example.tiff TIFF 108x160 108x160+0+0 16-bit DirectClass 105KB 0.000u 0:00.000

I want to apply a -level operation and then convert the resulting image to a 24-bit PNG:

Code: Select all

convert-hdr hdr_depth_clamp_example.tiff -level 9%,65%,1.5 -clamp -depth 8 convert-hdr-clamp-depth.png

This gives , obviously the darker-than-black has been wrapped to white for each channel, even though I specifically asked for -clamp.

I tried the exact same command with the plain Q16 imagemagick:

Code: Select all

convert hdr_depth_clamp_example.tiff -level 9%,65%,1.5 -clamp -depth 8 convert-clamp-depth.png

This gives , and looks as it should.

I did find a way around my problem by specifying PNG24:

Code: Select all

convert-hdr hdr_depth_clamp_example.tiff -level 9%,65%,1.5 PNG24:convert-hdr-png24.png

This gives .

Have I misunderstood the meaning of -clamp or is it a bug?

convert-hdr:

Version: ImageMagick 6.8.4-9 2013-04-12 Q16 http://www.imagemagick.org
Copyright: Copyright (C) 1999-2013 ImageMagick Studio LLC
Features: DPC HDRI OpenCL OpenMP
Delegates: bzlib djvu fftw fontconfig freetype jbig jng jp2 jpeg lcms lqr lzma openexr pango png ps rsvg tiff x xml zlib

convert:

Version: ImageMagick 6.7.7-10 2012-08-17 Q16 http://www.imagemagick.org
Copyright: Copyright (C) 1999-2012 ImageMagick Studio LLC
Features: OpenMP

[pipe]

Actually, using PNG24 with HDRI did not completely remove the problem, it just made it much less noticable.

This image is from using convert-hdr and saving as PNG24:



There are some color wraps, likely artifacts from a -resize 25% that I did on the image.

[anthony]

Could your image have transparent areas?

[pipe]

No, identify reports TrueColor, and it's straight from the scanner so it's unlikely. I did find the source of the bug though, and it's internal to ImageMagick and relates to how Not-a-Number floats are handled.

I performed some additional tests. I got the same thing to break using rose:

convert-hdr rose: -level 20,100%,1.5 -clamp -depth 8 x:

After having performed a lot of tests with -level, -clamp, and HDRI vs. non-HDRI ImageMagick I think I have found that it is the -level operator that breaks here, and more explicitly, it breaks iff these conditions are true:

• convert is compiled with HDRI support.
• The -level operator drives a dark pixel negative. (convert rose: -level 20%,100%,1.5...)
• The -level operator is applying a gamma != 1.0. (convert rose: -level 20%,100%,1.5...)
• The image depth is then set to 8 bpc. (convert rose: -level 20%,100%,1.5 -clamp -depth 8...)

So I got a bit curious, and dug through the source code to find the reason for this. The culprit is the gamma operation in magick/enhance.c/LevelPixel(), which does a call to pow() with a negative value due to the stretch, thus returning -NaN (Not a Number). Nothing similar happens with bright pixels, they just return a very large but valid number.

This -NaN is stored in the image and propagated through most other functions, since almost every operation on a NaN results in a NaN. Clamping the image with -clamp doesn't help, because the NaN is propagated.

Then we reach magick/attribute.c/SetImageChannelDepth() which for the HDRI version calls ScaleAnyToQuantum(ScaleQuantumToAny()) for every pixel in every channel. ScaleQuantumToAny() actually converts the HDRI floats to an integer.

This is where the silly things happen. On my platform, gcc 4.7.2 on a Core2 Quad, running 64-bit Ubuntu, casting a NaN to an integer that is 4 bytes or less returns 0. Casting a NaN to an integer larger than 4 bytes returns 9223372036854775808. This is exactly why the program breaks when I use -depth 8 but doesn't break as much when I use PNG24. PNG24 doesn't use ScaleQuantumToAny, but rather ScaleQuantumToChar. This turns the NaN pixels black instead of white, and thus it makes the error invisible. ScaleQuantumToAny casts it to a large integer, likely 8 bytes, and thus the pixel gets a really huge value instead of 0, so it turns white.

I'm not sure what the best solution is. This problem is likely to happen everywhere gamma correction is taking place on pixels with negative values, so fixing it may be difficult. One option is to check for -Nan and +NaN in magick/threshold.c/ClampPixel(). -NaN would be black, +NaN would be white.

[magick]

We can reproduce the problem you posted and have a patch in ImageMagick 6.8.5-0 Beta available by sometime tomorrow. Thanks.

[anthony]

Seems strange...

But then to me gamma is strange.


What about conversion from RGB -> sRGB with an image containing a negative number (say due to a level in HDRI).

[snibgo]

Excellent detective work by pipe.

RGB <->sRgb conversions should be okay if the usual formula is extended to negative numbers, as the bottom part of the slope is linear.

[pipe]

For more headache-inducing problems, try a gamma of exactly 0.5 involving negative pixel values and a HDRI enabled ImageMagick:

convert-hdr rose: -scale 400% -level 50,100%,0.5 -depth 8 x:

This flips the negative values to positive, because of mathemagic. (Because (-x)^y is defined when y is an integer, and a gamma of 0.5 gives an exponent of 1/0.5=2)

I made a graph that tries to visualize the problem:


There's no good way to deal with the problem of gamma-correcting negative values that I know of. The three solutions in the image are:

• Set every negative value to zero. (magenta)
• Leave the negative values as-is. (red)
• Apply gamma correction to the magnitude of the pixels, while keeping the sign. (blue)

None of these are mathematically sound, and they all have different drawbacks. Two other options I can think of:

• Set these pixels to the negative infinite. This would mathematically be in line with the 'correct' gamma curve, but is also weird.
• Treat all pixel values as complex numbers instead of reals. This would be crazy.

Maybe the wise wizards can solve this in a better way.

[magick]

The wise wizards decided to check if the result of the pow() function is NaN, we replace it with a zero. If you decide there is a better solution, let us know.

[pipe]

One problem with the current solution is if someone tries a gamma of 0.5. This will invert every negative value. I can see how this could be troublesome, because most of those pixels will still have a very dark color, so they will not be detected by the naked eye under many viewing conditions. They may cause a surprise later in the processing chain when it's already too late.

Personally I think I would test the value for < 0.0 before applying the pow() function, to reduce the risk of anyone getting negatively surprised.

If the pixel is negative, the only two sane options are to either set it to 0.0, or to leave it alone. I think I would prefer if they are left alone. It would preserve more information, and it still leaves the user with the option of doing a -clamp if the negative values will cause a problem later on in the chain.

I took the freedom of testing this in the code, and it seems to work great. Here's a patch for magick/enhance.c that works with both -levels and +levels:

Code: Select all

Index: magick/enhance.c
===================================================================
--- magick/enhance.c	(revision 11960)
+++ magick/enhance.c	(working copy)
@@ -2814,6 +2814,10 @@
 %      use 1.0 for purely linear stretching of image color values
 %
 */
+static inline double gamma_pow(const double value,const double gamma)
+{
+  return(value<0.0 ? value : pow(value,1.0/gamma));
+}
 
 static inline double LevelPixel(const double black_point,
   const double white_point,const double gamma,const MagickRealType pixel)
@@ -2823,9 +2827,8 @@
     scale;
 
   scale=(white_point != black_point) ? 1.0/(white_point-black_point) : 1.0;
-  level_pixel=QuantumRange*pow(scale*((double) pixel-
-    black_point),1.0/gamma);
-  return(IsNaN(level_pixel) != MagickFalse ? 0.0 : level_pixel);
+  level_pixel=QuantumRange*gamma_pow(scale*((double) pixel-black_point),gamma);
+  return(level_pixel);
 }
 
 MagickExport MagickBooleanType LevelImageChannel(Image *image,
@@ -3011,7 +3014,7 @@
 {
 #define LevelizeImageTag  "Levelize/Image"
 #define LevelizeValue(x) (ClampToQuantum(((MagickRealType) \
-  pow((double)(QuantumScale*(x)),1.0/gamma))*(white_point-black_point)+ \
+  gamma_pow((double)(QuantumScale*(x)),gamma))*(white_point-black_point)+ \
   black_point))
 
   CacheView

[fmw42]

FYI. I recently revisited one of my scripts and found that I had added -clamp with +levels (for use in HDRI). So I have seen this issue before. As long as there is the ability to add -clamp to control out of bounds situations (which may be meaningless anyway), then leaving it to the user to clamp is OK with me. As I recall we added (or at least discussed) a patch to -evaluate log to automatically clamp, but I am not positive of that. There may be one or two other functions that were internally clamped so as to reproduce results to match non-HDRI situations. They may have to be reviewed. The changelog shows many of the changes that occurred for clamp.

[anthony]

I would apply the patch.. leave negative numbers as is. Let the user clamp as needed (automatic for most non-HDRI image file formats)

[magick]

The patch is in ImageMagick 6.8.5-0 Beta. Thanks.

[anthony]

Sorry slight correction. leave negative numbers as the 'leveled' value, without the gamma correction.
But that does seem to be what the patch does, by using a in-line function.

You may also need to look at the reverse operation, +level too, as larger than 0-1 ranges in that case can also produce negative numbers, or the input could be negative before hand.

[pipe]

Yeah, what the patch does is simply to test for negative numbers just before applying the operation that would cause negative numbers to break. The level operation is left intact.

This is however not the case with the -gamma operator, which works through a lookup-table, which has the effect of both quantizing any HDR floats to 16 bits, and also clipping them to 0.0 - 1.0.

So at the moment, if you just want to gamma-adjust a HDR image while retaining as much quality and information you should use -level 0x100%,gamma. It will be slower.

This command creates a grayscale ramp, expands the levels a lot, apply a gamma operation and then reduces the levels to the original range again.

convert-hdr -size 100x300 'gradient:gray(0)-gray(100%)' \( +clone -level 40x60% -level 0x100%,2 +level 40x60% \) \( 'gradient:gray(0)-gray(100%)' -level 40x60% -gamma 2 +level 40x60% \) +append x:

As you can see, the -gamma operator to the far right clamps the values, while the -level operator doesn't. You can also see that it passes the non-gamma-adjustable pixels through without modification.