EXR data point accuracy

[seanburke1979]

Hello all.
I am experimenting with the floating point image format EXR. I can write and read from the format, but I am getting horrible fidelity. I will present my results first and then attach my benchmark code at the end. I apologize in advance for the long post.

The attached code, exr_test.c, starts with a 3x3 array filled with the value 1.11111111. From there, it writes the values to an exr file and then reads the exr file. No explicit arithmetic is performed. Here are the results for a q8-HDRI build:

Code: Select all

        Benchmark Array:                        Read-in array:                  Difference:
[1.11111111 1.11111111 1.11111111 ] [1.11087799 1.11087799 1.11087799 ] [0.00023312 0.00023312 0.00023312 ]
[1.11111111 1.11111111 1.11111111 ] [1.11087799 1.11087799 1.11087799 ] [0.00023312 0.00023312 0.00023312 ]
[1.11111111 1.11111111 1.11111111 ] [1.11087799 1.11087799 1.11087799 ] [0.00023312 0.00023312 0.00023312 ]

...for a q16-HDRI build...

Code: Select all

        Benchmark Array:                        Read-in array:                  Difference:
[1.11111111 1.11111111 1.11111111 ] [1.10935807 1.10935807 1.10935807 ] [0.00175304 0.00175304 0.00175304 ]
[1.11111111 1.11111111 1.11111111 ] [1.10935807 1.10935807 1.10935807 ] [0.00175304 0.00175304 0.00175304 ]
[1.11111111 1.11111111 1.11111111 ] [1.10935807 1.10935807 1.10935807 ] [0.00175304 0.00175304 0.00175304 ]

and finally, for a q32-HDRI build:

Code: Select all

        Benchmark Array:                        Read-in array:                  Difference:
[1.11111111 1.11111111 1.11111111 ] [1.10935807 1.10935807 1.10935807 ] [0.00175304 0.00175304 0.00175304 ]
[1.11111111 1.11111111 1.11111111 ] [1.10935807 1.10935807 1.10935807 ] [0.00175304 0.00175304 0.00175304 ]
[1.11111111 1.11111111 1.11111111 ] [1.10935807 1.10935807 1.10935807 ] [0.00175304 0.00175304 0.00175304 ]

So the error is constant in the 16 and 32 bit versions of IM. Additionally, small values such as 1E-3 disappear completely. So here is my question: is the error in the EXR coder, backup library or is it simply not spec'ed for true double precision floating point values?
Here is the test code:

Code: Select all

#include <stdlib.h>
#include <string.h>
#include <magick/MagickCore.h>

void snap( unsigned long x, unsigned long y, double *out, char *fname){
	ExceptionInfo 
		*exception = AcquireExceptionInfo();
	Image
		*image;
	unsigned long i,j;
  	MagickPixelPacket
	 pixel;
        PixelPacket *q;
  	ImageInfo
	 *image_info= CloneImageInfo((ImageInfo *)NULL);
	
  	QueryMagickColor("black", &pixel, exception);

	strcpy(image_info->filename, fname);
	image = NewMagickImage(image_info, x, y, &pixel);

	for(j = 0; j < 3; j++){
		q = SetImagePixels(image, 0, j, 3, 1);
		for(i = 0; i < 3; i++){
			q->red = (Quantum)(out[i + 3 * j]);
			q++;
		}
		if (SyncImagePixels(image) == MagickFalse)
			break;	
	}
	WriteImage(image_info, image);
}

int main(){
 double test_arr[9] = { 1.234567E-2, 1.11111111, 1.11111111,
		 1.11111111, 1.11111111, 1.11111111, 
		1.11111111, 1.11111111, 1.11111111},
   return_arr[9];
  ExceptionInfo
    *exception;
  Image
    *in;
  ImageInfo
    *info;
  PixelPacket *q;
  unsigned long i, j;
  char fname[20]; 

  MagickCoreGenesis(0,MagickTrue);
  exception = AcquireExceptionInfo();
  info = AcquireImageInfo();
  sprintf(fname, "exr_test_%lu.exr", MAGICKCORE_QUANTUM_DEPTH);
  snap(3,3, test_arr, fname);
  strcpy(info->filename, fname);
  in = ReadImage(info, exception);

  for( j = 0; j < 3; j++){
   q = GetImagePixels(in, 0, j, 3, 1);
   for(i = 0; i < 3; i++){
    return_arr[i + j * 3] =(double)q->red;
    q++;
   }
  }
  printf("\tBenchmark Array:\t\t\tRead-in array:\t\t\tDifference:\t\n");
  for(i = 0; i < 3; i++){
   printf("[");
   for(j = 0; j < 3; j++){ printf("%1.8e ", test_arr[j + i*3]); }
   printf("] [");
   for(j = 0; j < 3; j++){ printf("%1.8e ", return_arr[j + i*3]); }
   printf("] [");
   for(j = 0; j < 3; j++){ printf("%1.8e ", test_arr[j + i*3] - return_arr[j + i*3]); }
   printf("]\n");
 }
return 0;
}

[magick]

EXR stores pixel data as 16-bit half floating point values so I would expect some loss of precision. If this does not account for the loss of precision you posted about, let us know and we will revisit the problem.

[seanburke1979]

Hmmm. Ok. Is there a true double precision FP format out there?

[magick]

ImageMagick supports doubles for MIFF, MPC, PFM, and TIFF when HDRI is enabled.

[fmw42]

Magick:

As background, these posts relate to Sean's FFT development and my testing showing poor quality results with floating point outputs for certain "weak signal" images, using HDRI. Compiling as non-hdri allowed us to get only so far with certain kinds of processing from the FFT images. Other kinds of processing on the FFT images seem to need full floating point formats. Thus we have been attempting to work with HDRI and have not had good success so far. EXR does support a full floating point version, I believe, in addition to the half format. Any chance, if this becomes needed, that IM can add that format for HDRI? We would certainly like to save the results in a non-IM-specific format, such as EXR or TIFF. We will try using TIFF or PFM in floating point format with HDRI and see what we get. We will also try MIFF and MPC and see what results, too.

Thanks.

Fred

[magick]

If you can point us to a method in the OpenEXR delegate library that supports full float or double values, let us know and we will get support into a future release of ImageMagick. The method we use is ImfInputReadPixels() which returns half's and we convert these to floats with the ImfHalfToFloat() method.

[fmw42]

Not being much of a programmer, I cannot identify a specific method (perhaps Sean can). But from the EXR page http://www.openexr.com/index.html it says:

"Support for 16-bit floating-point, 32-bit floating-point, and 32-bit integer pixels. The 16-bit floating-point format, called "half", is compatible with the half data type in NVIDIA's Cg graphics language and is supported natively on their new GeForce FX and Quadro FX 3D graphics solutions."

code available at:
http://www.openexr.com/downloads.html

With regard to floating point TIFF, I have been searching the Internet for any information. The TIFF 6.0 Specs do not really mention it as far as I can tell. I did find a brief statement in a TIFF FAQ about

"Dynamic range Grayscale or color imagery:
colorspace floating point Grayscale or RGB
compression LZW or Deflate"

In Q(8,16,32-bit) HDRI IM, will an output to TIFF automatically be put into floating point format and preserve the fractional components and negative values?


Fred