Different behaviour between installation versions

[robert_a]

HI,
I have a command that was working to give the correct ressult, I have compiled the current version of ImageMagick and now it gives a wrong result.
The command converts a 32-bit floating point TIFF in which the values lie in the range 0-65535 into a 16-bit TIFF in which the values are the truncated integers. I need to use a self-compiled Q32 installation for this purpose.
The 'wrong result' looks like it's quantized to only 8 grayscale "colours" and randomly assigned values, the 'right result' has integers equal to the floating-point values but truncated.
Any help, what has changed?



Here is the command:

convert -define quantum:format=floating-point projections/p_00000.tif -evaluate divide 65536.0 -define quantum:format=integer -depth 16 image_6897.tif

Old version of ImageMagick covert that gives the result I want:
8]$ convert -version
Version: ImageMagick 6.7.5-10 2012-03-05 Q32 http://www.imagemagick.org
Copyright: Copyright (C) 1999-2012 ImageMagick Studio LLC
Features: OpenMP HDRI


New version of ImageMagick convert that gives a different result -- looks like it is quantized to 8 bit then wrapped around the negative values??

]$ convert -version
Version: ImageMagick 6.8.9-7 Q32 x86_64 2014-09-15 http://www.imagemagick.org
Copyright: Copyright (C) 1999-2014 ImageMagick Studio LLC
Features: DPC HDRI OpenMP
Delegates: bzlib fftw fontconfig freetype jng jpeg lcms pangocairo png tiff x xml zlib

[snibgo]

Correct me if I'm wrong, but I think you want to take an input file that is Q32 floating point, and make an output that is Q16 integer, with effectively the same values. Zero becomes zero, 4 billion becomes 65535, etc. Is that right?

If so, the first command is wrong. Perhaps it relies on a bug in that old version. No need to divide the values; IM should do that for you. I would expect this to work:

Code: Select all

convert projections/p_00000.tif -depth 16 image_6897.tif

If that doesn't work, can you provde a sample input file? You can put it somewhere like dropbox.com and paste the URL here.

[snibgo]

Ah, I was wrong. Sorry. I've just seen you said, "in which the values lie in the range 0-65535".

So, when the input value is 65535 (out of 4 billion), what do you want the output to be? 65535 out of 65535, or something else?

[robert_a]

Not quite, the values are float not 32bit Int , so the value is 0.000000 up to 65534.999999 out of the whole range of 32bit Floating point values. That is because they result from a calculation in which it is desirable to carry the extra precision. However, another image processing step I want to try is not (yet) programmed for 32-bit floating point TIFF files as input, and the prior program is not (yet) programed to truncate the output into 16 bit (may not be desirable in the long run)


So I wish 0.0000 to be mapped to 0
0.1 to be mapped to 0
...
1.0 to be mapped to 1
1.1 to be mapped to 1
1.99 to be mapped to 1
2.0 to be mapped to 2
65534.99 to be mapped to 65534

The command I posted originally, with a Q32 build and specifying quantum float and also division by 65536 before conversion to 16-bit Int , simply did this in that particular build of ImageMagick "convert". It does not do this in the new build of ImageMagick convert. I suspect it is something to do with 'Endian-ness' in a way that does not make sense to me, since the 'endian-ness' of the TIFF file is well defined by the TIFF header. (little-endian in this case) Has the default 'endian' behaviour for such calculations changed since that previous version?

[robert_a]

Note, also I beleive you are not correct about the division behavior, it seems clear that ImageMagick's conversion from floating point to another type assumes that the values are in the range of 0-1 and maps this range to the full range of the output type. It seems you are thinking of 32 bit integer types .


I can live with that, though I think it is not efficient for my case, rethinking the earlier- and later- steps may solve that but IM should help to do that re-thinking!

[snibgo]

As I understand it, an input quantum should map to an output quantum, both irrespective of HDRI.

So an input Q32 value of 4294967295 (or HDRI 4294967295.0) should map to an output Q16 65535 (or HDRI 65535.0), without any intervention by us. A Q32 input value of 65536 will map to an output Q16 value of 1, more or less. This would be with:

Code: Select all

convert in32.tiff -depth 16 out16.tiff

You want greater values in the output, so you need to multiply the values first:

Code: Select all

convert in32.tiff -evaluate multiply 65536 -depth 16 out16.tiff

This means the in-built Q32->Q16 conversion will work as you want, more or less.

However, IM doesn't convert from Q32 to Q16 by merely dropping the last 16 bits. (Nor does it convert from Q16 to Q32 by merely adding 16 zero bits.) So the arithmetic probably needs to be more complex than "-evaluate multiply 65536".

You might create a simple sample file, containing only five pixels or so, and experiment.

[robert_a]

That command results in an image with value 65535 everywhere.

[snibgo]

Can you provide a sample input file? You can put it somewhere like dropbox.com and paste the URL here.

[robert_a]

The results of the command originally mentioned:

convert -define quantum:format=floating-point input_image.tif -evaluate divide 65536.0 -define quantum:format=integer -depth 16 image_from_version_6.7.5.10.tif

and the same command using version 6.8.9.7 ; along with the input image, are on the ftp site for a couple of weeks until they are auto deleted --


ftp://ftpanon.diamond.ac.uk/I12JEEP/ForImageMagick/

The file names : ' input_image.tif ' , the input image , 32-bit IEEE float TIFF file
'image_from_version_####.tif' -- the output image when the command is run with the specified version (compiled here)

The file
image_from_version_6.7.5.10.tif
contains the image I require.

The file
image_from_version_6.8.9.7.tif
contains the peculiar result with only 8 different values distributed among the 65536 possible values in a way that is not obvious to me.

Using the command recommended by snibgo results in an image that appears to contain the value 65535 everywhere (both versions)
I put these on the FTP also , as 'image_snigbo_6.8.9.7.tiff' etc.

Thanks for any further clarifications of imagemagick behaviour!

[snibgo]

Not quite, the values are float not 32bit Int , so the value is 0.000000 up to 65534.999999 out of the whole range of 32bit Floating point values.

But when I examine input_image.tif:

Code: Select all

%IM32f%identify -verbose input_image.tif

I get:

Code: Select all

  Channel statistics:
    Gray:
      min: 0 (0)
      max: 1.87475e+13 (4365)
      mean: 3.77842e+12 (879.731)
      standard deviation: 2.69001e+12 (626.316)
      kurtosis: -1.1503

The mean value is about 3 million million, which is substantially above your claimed 65534.999999.

[robert_a]

okay .. but it's ImageMagick that is telling you this? I placed a screenshot of ImageJ opening that same file displaying the value at a test point of 2017.0 as expected (in the same FTP folder)


Why did I get the desired output in version 6.7.5-10 ? This is the burning question. If the image is 'wrong' why did it work before?


Why does other software, ImageJ, report the original image as having the values I believe that I have put in the image but ImageMagick 6.8.9-7 reports different values??

Thanks for looking at this. I think it is more subtle that at first suspected.

[robert_a]

But when I examine input_image.tif:

Code: Select all

%IM32f%identify -verbose input_image.tif

I get:

Code: Select all

 Channel statistics:
Gray:
min: 0 (0)
max: 1.87475e+13 (4365)
mean: 3.77842e+12 (879.731)
standard deviation: 2.69001e+12 (626.316)
kurtosis: -1.1503

The mean value is about 3 million million, which is substantially above your claimed 65534.999999.

What is the meaning of the values in parenteses after the 3million million?? these values, 4365, 879.731 , are the actual maximum and mean of the values I beleive are in this particular image. These are the values that I want to appear (truncated in 16 bit form) in the output image.

Using Matlab as well reads teh values I expect --

inputim=imread('input_image.tif');
max(max(inputim))
ans = 4365

mean(mean(inputim))
ans = 879.7311

[snibgo]

The number in parentheses is the pixel value expressed as a multiple of (quantum-1). For Q32, quantum is 2^32. For integer IM, this value in parentheses will be between 0.0 (black) and 1.0 (white). For HDRI, the value can be outside these limits. A factor of 4365 is substantially outside. We could divide all the values by 4365 to get them within the normal range, so the image would be visible.

Your desired result is:

Code: Select all

%IM32f%identify -verbose image_from_version_6.7.5.10.tif

  Channel statistics:
    Gray:
      min: 0 (0)
      max: 4365 (0.0666056)
      mean: 879.704 (0.0134234)
      standard deviation: 626.316 (0.00955697)
      kurtosis: -1.1503
      skewness: 0.504835

The maximum value in input_image.tif is 4365 multiplied by quantum. You want it to be 0.0666056 times quantum. So we need to divide by 4365/0.0666056 = 65536 (more or less).

Code: Select all

%IM32f%convert input_image.tif -evaluate divide 65536 -depth 16 -define quantum:format=integer out.tiff

Check the result is as required:

Code: Select all

%IM%compare -metric RMSE image_from_version_6.7.5.10.tif out.tiff NULL:

0 (0)

The result is as required.

[robert_a]

Snibgo said:,

So we need to divide by 4365/0.0666056 = 65536 (more or less).

That is what I said in the first place before the discussion about the handling of quantization.

Here is the command:

Code: Select all

convert -define quantum:format=floating-point projections/p_00000.tif  -evaluate divide 65536.0 -define quantum:format=integer -depth 16 image_6897.tif

YOur command:

Code: Select all

convert input_image.tif -evaluate divide 65536 -depth 16 -define quantum:format=integer out.tiff

Apart from an initial -define quantum:format which maybe unnecessary, this is the same. And it produces the same results on my system with my build of version 6.8.9-7 which differ from the result of the earlier version.

Snibgo, Can you open the resulting tiff in ANOTHER software such as (free) imageJ ? What values are found? Can you
(1) put that file where I can get it
(2) report what your 'convert' version and build options are ?

Should we move this discussion to 'bugs' forum? I am convinced this is not correct behaviour but a bug

[snibgo]

Code: Select all

%IM32f%convert input_image.tif -evaluate divide 65536 -depth 16 -define quantum:format=integer from32f_out.tiff

%IM%identify from32f_out.tiff
from32f_out.tiff TIFF 2560x1035 2560x1035+0+0 16-bit Grayscale Gray 5.308MB 0.000u 0:00.000

%IM%convert from32f_out.tiff -crop 1x1+1200+500 txt:
# ImageMagick pixel enumeration: 1,1,65535,gray
0,0: (2.78019%,2.78019%,2.78019%)  #071E071E071E  gray(2.78019%)

Gimp shows from32f_out.tiff pixel at (1200,500) has value 7 (out of 255). This is consistent with the single crop from convert, which shows #071E, 2.78019%.

Versions:
%IM% is the binary distribution v6.8.9-5 for Windows (8.1). Q16, non-HDRI.

%IM32f% is v6.8.9-6 Q32 with HDRI, compiled by me with Cygwin for Windows.

I don't have the current version, v6.8.9-7.

If you think there is a bug in the current version, feel free to raise it. I can't see a bug.

from32f_out.tiff:
https://dl-web.dropbox.com/get/from32f_ ... SvULA&dl=1