Legacy ImageMagick Discussions Archive

Hi,

I'm using Magick++ to read FITS files. I have lots of test images from Fitswork, PixInsight, ImageTOOLScs and ImageMagick itself.

Currently I'm testing the 16 bit signed format (BITPIX=16). In this case you normally have (like e. g. PixInsight does)

BZERO=32768
BSCALE=1

This achieves that the signed integer value contained in the FITS file is added to 32768 and then multiplicated with 1. The result is an unsigned 16 bit integer (0..65535) that can be used to represent the image data internally in a software.

For debugging purposes I created an image containing 256 gray values distributed from 0..65535. After the following code (simplified)

Code: Select all

Image img;
img.read(path);
Magick::Pixels* view = Pixels(img);
int width = img.columns();
int height = img.rows();
PixelPacket* pixels = view->get(0,0,width, height);

... the image data is in the array pixels, containing 8 bytes for each pixel (BGRA).

As said, my test FITS image contains values from -32768 to 32767 which I verified using a hex editor. When reading this test image with the above code, the array contains no values below 0x5555. The values are distributed from 0x5555 (which was black in my test image) to 0xFFFF. This makes the image considerably brighter (less contrast). This occurs independent from other tags in the header. Googling this problem I found that people simply stretch FITS images after reading them to compensate this bug. But there's loss of data (brightness resolution). In addition it is not guaranteed that stretching the image generates the same image as the original image.

My current workaround is that I make a temporary copy of the image, verify that it is of type BITPIX=16 and then patch the FITS header by setting BZERO=21846 (0xAAAA / 2) and removing any DATAMIN or DATAMAX entries. When I read in such a patched image with the above code, the image data is all well. This works with images from PixInsight, Fitswork and ImageMagick itself (ImageMagick cannot read in correctly its own FITS files for BITPIX>=16). But this is just a workaround, I'm sure there are cases where it does not work (for example if BSCALE is different from 1 in the input image).

Its very simple to reproduce this bug. Convert any 16 bit TIF image to FITS (with ImageMagick), then convert it back to TIF and it will be lighter. Reason is the above described problem. But be sure not to use IrfanView, because IrfanView stretches images. Use Fitswork, Photoshop or - best - a hex editor. The lowest value will be 0x5555.

Kind regards
Stefan

It's very likely the sRGB versus RGB colorspace problem. Grayscale images by default are linear RGB while color are sRGB, if they aren't tagged with a colorspace.

to follow up with glenn advice, see http://www.imagemagick.org/script/color-management.php

Unfortunately, colorspace is not the problem. If it was, image data would be non linear and would start at 0 for black. But in fact it is linear and starts at 0x5555 (which was black=0x8000 in the FITS file, but now some gray value because of the bug). But forget theory, just make a test:

Code: Select all

convert test.fits test.tif

generates the same (wrong) result as

Code: Select all

convert test.fits -colorspace RGB test.tif

On the contrary if I write

Code: Select all

convert test.fits -colorspace sRGB test.tif

... then test.tif looks even brighter than with RGB. All wrong. Always keep in mind that test.fits contains no colorspace in its header - and using a hex editor I see that it contains linear (colored) image data.

Though I'm no ImageMagick expert, it seems to be obvious to me that this is indeed a bug? It would be really great if someone gave me a hint where I have to search in coders\fits.c to fix that (somewhere in function ReadFITSImage ?) I think ImageMagick is a great library, but this problem with FITS has been discussed since years. As a workaround, people stretch FITS images after reading. This is no solution, because obviously you need the original image data (which might be not stretched).

By the way, this bug does not occur with 8 bit images. And I'm using ImageMagick 6.7.9. I uploaded my test.fits here: https://www.dropbox.com/s/6puld2w6woxrlnn/test.fits (you need to left click that, otherwise you will get html code). Please keep in mind that this problem occurs not only with this single sample image (and its specifics), but occurs with all of my 16 bit FITS images written by Fitswork, PixInsight, ImageTOOLSca and ImageMagick itself.

I know nothing about FITS format.

If I take a greyscale 16-bit tiff file that uses most of the range from black to white, created from a photograph, convert it to fits and back ...

Code: Select all

convert g.tiff f.fits
convert f.fits f2.tiff

... then f2.tiff is lighter than g.tiff. The shadows have become lighter, while the highlights have become darker. According to identify verbose, this shift has occurred in the first conversion.

Code: Select all

g.tiff: 
    Gray:
      min: 1029 (0.0157015)
      max: 64694 (0.987167)
      mean: 15058.7 (0.229781)
      standard deviation: 13962.5 (0.213054)
      kurtosis: 1.60661
      skewness: 1.6557

f.fits:
    Gray:
      min: 16898 (0.257847)
      max: 48731 (0.743587)
      mean: 23913.1 (0.36489)
      standard deviation: 6981.24 (0.106527)
      kurtosis: 1.60661
      skewness: 1.6557
f2.tiff:
    Gray:
      min: 16898 (0.257847)
      max: 48731 (0.743587)
      mean: 23913.1 (0.36489)
      standard deviation: 6981.24 (0.106527)
      kurtosis: 1.60661
      skewness: 1.6557

This looks like a bug to me, not a RGB/sRGB issue.

We'll take a look. Most likely the colorspace is not being defined to gray which does not include a gamma function whereas sRGB does.

The problem is linear, not gamma. A workaround:

Code: Select all

convert g.tiff f.fits

Code: Select all

convert f.fits -evaluate Subtract 25% -evaluate Multiply 2 f2.tiff

Now:

Code: Select all

D:\web\im>"%IMG%compare" -metric RMSE g.tiff f2.tiff NULL:
0.707131 (1.07901e-005)

Trying the subtract and multiply in the conversion to fits still gives a file that ranges from 25% to 75%.

We can reproduce the problem you posted and have a patch in ImageMagick 6.8.2-0 Beta available by sometime tomorrow. Thanks.

Hi,

thanks for the upgrade, I just gave it a try. In fact the image data changed. But it is still wrong. Now, images are not too bright any more, but too dark.

For a test I used a 16 bit tif image you can download here: https://www.dropbox.com/s/bvl89thijf2jqbo/test.tif

If you write:

Code: Select all

convert test.tif test.fits
convert test.fits test2.tif

... the resulting image is too dark. Using Fitswork I saw that test.fits is ok, the problem still is reading FITS images.

Kind regards,
Stefan

We can reproduce the problem you posted and have a patch in ImageMagick 6.8.2-0 Beta available by sometime tomorrow. Thanks.

Hi,

thanks for the upgrade. I downloaded

Code: Select all

ImageMagick-6.8.2-0-windows.7z	26-Jan-2013 19:39

and compiled it, which resulted in 6 build errors, the first being in magick\distribute-cache.c (MSG_NOSIGNAL is undeclared).

Since IM_MOD_RL_fits_.dll did compile successfully, I copied all DLLs which were built successfully into my application directory (MAGICKCORE_INSTALLED_SUPPORT is undefined).

When writing

Code: Select all

convert test.tif test.fits
convert test.fits test2.tif

test2.tiff is still much brighter than test.tif (https://www.dropbox.com/s/bvl89thijf2jqbo/test.tif).

Kind regards,
Stefan

Hi,

I just tried the new beta version 6.8.2-2 from 29-Jan-2013 13:47, compiled it successfully. Still the same problem, that means after

convert test.tif test.fits
convert test.fits test2.tif

... test2.tif is much brighter than test.tif. My 16 bit test image is here: https://www.dropbox.com/s/bvl89thijf2jqbo/test.tif (use left click otherwise you download html code)

A little preview:

Kind regards,
Stefan

Hi,

thanks for your version 6.8.2-2 from 30-Jan-2013 20:53. If you read a FITS image which was created by ImageMagick, the problem is solved now.

But FITS supports a couple of sub formats. Loading 16 Bit Uint FITS images still does not work, Images become much brighter when being loaded with ImageMagick.

Here I saved three times the same image using PixInsight, Fitswork and ImageVIEWca using 16Bit Uint FITS: https://www.dropbox.com/sh/7svllett3d4nnad/ZrCYIuP9En - all three images become much brighter than the original input.tif when being converted with

Code: Select all

convert 16BitUint.fits test.tif

Kind regards,
Stefan

Hi,

thanks for your version 6.8.2-3 from 03-Feb-2013. Reading the UInt16 sample FITS images from my previous posting now works better. After converting these sample images to tif, they look similar to the original images. Here I used two FITS test images, one unsigned 16 bit integer and the other one signed 16 bit integer (see my previous posting for a link to the uint16):

But still their brightness is not correct. In addition the current ImageMagick version can't even write FITS correctly. After writing

Code: Select all

convert test.tif test_im.fits

the resulting test_im.fits looks like this:

Using Fitswork I verified that the problem is really inside the fits image, not in imdisplay. Writing FITS images did work in the last versions, but no more in today's ImageMagick version.

Kind regards,
Stefan

Classic, fix one bug, introduce another. We can reproduce the problem and will have a patch soon in ImageMagick-6.8.2-4 Beta.

Legacy ImageMagick Discussions Archive

FITS images become too bright

FITS images become too bright

Re: FITS images become too bright

Re: FITS images become too bright

Re: FITS images become too bright

Re: FITS images become too bright

Re: FITS images become too bright

Re: FITS images become too bright

Re: FITS images become too bright

Re: FITS images become too bright

Re: FITS images become too bright

Re: FITS images become too bright

Re: FITS images become too bright

Re: FITS images become too bright

Re: FITS images become too bright

Re: FITS images become too bright