Legacy ImageMagick Discussions Archive

Dear folks, I found that IM converter TO jp2 format seems to change (in some sense lose) data. However, I could not find a related discussion, so posting my findings.
First, I'm using linux gentoo x64 and
ImageMagick 6.9.0-3 Q16 x86_64 2016-04-28
Second, I need INDEED a lossless transformation. And, accounting the compression success of jp2 I would prefer jp2 given I have already about 3TB of tiff-s. Again, lossless means strictly mathematically lossless. No single bit should change.
After trial and tries, the test I did is as follow:
0. I took a Bliss.bmp file, standard windows xp wallpaper.

So, kind of a random source. Then:
1. I converted it to png using IM (im.png)
2. I converted it to jp2 using IM saying "-quality 100" (I tried also "-compress lossless" to no avail, the method is compiled in but the result is the same as "-quality 100", i.e. jp2-files are identical) (im.jp2)
3. I converted im.png to impng.bmp
4. I converted im.jp2 to imjp2.bmp
5. I compare impng.bmp and imjp2.bmp using "cmp -l" and see that actual bitmaps are different. 214 bytes differ.
So, at this stage it is obvious that that some conversions change data.

To find out which I did as follows:
6. I converted Bliss.bmp to png using photoshop (ps.png)
7. I converted Bliss.bmp to jp2 usnig photoshop (ps.jp2)
8. I converted ps.png to pspng.bmp using photoshop
9. I converted ps.jp2 to psjp2.bmp using photoshop
10. I compared bitmaps of pspng.bmp and psjp2.bmp and they are identical
So, photoshop at least pretends to be lossless.

Thus, I did cross-checks.
11. I converted photoshop produced png and jp2 to bmp-s using IM. The names are pspngim.bmp and psjp2im.bmp.
12. I checked that all 4 bitmaps (pspngim.bmp and psjp2im.bmp and pspng.bmp and psjp2.bmp) Of course I carefully accounted that headers from photoshop and IM are different and used the offset from the header. Then I specified an appropriate SKIP1 SKIP2 to "cmp -l" command.
So, it seems that conversion FROM jp2 are consistent in IM.

Further I did like that:
13. I converted IM produced png and jp2 to bmp-s using photoshop. The names are impngps.bmp and imjp2ps.bmp
14. Checking bitmaps I got that:
bitmaps of impng.bmp and impngps.bmp coincide
bitmaps of imjp2.bmp and imjp2ps.bmp coincide
bitmaps of impng.bmp and imjp2.bmp (or equivalently impngps.bmp and imjp2ps.bmp) differ in 214 bytes.

Finally the grand crosscheck was done, the comparison shows that:
15. bitmaps of impng.bmp and pspng.bmp coincide

From here the conclusion is that data change as long as a conversion by IM in the direction TO jp2 is involved. Since it is exactly what I need I'm seeking for help/advice.
I accept that conversion results are indistinguishable by a human eye. But, for my needs I need just strictly invertible transforms. I can go with photoshop but I'm in favour of opensource and batch command line processing (preferably in linux).

All referred files are here: http://altaray.net/inout/upload/ll.tar.bz2

I guess my programming experience is enough to find out the glitch but the source is large. I'd be happy to be pointed more or less precisely to where I should look at. Math aspects are not a problem as I'm a theoretical physicist, ii.e. math is the simplest part for me.

Thanks, Alex

Metric AE shows the number of pixels that have changed. Here, zero. Not a single bit of image data has changed.

EDIT to add: Of course, you should distrust the use of ImageMagick to verify that ImageMagick correctly processes JP2. I expect Photoshop can compare images and tell you whether or not they are identical, to confirm that IM is correct.

If you want to ensure that not only are the images identical, but a round-trip creates an identical file (including the metadata), that job is much harder.

Dear snibgo, In my case I have this: i.e. bmp and jp2 differ in 177 points.
bmp and png are identical.

SO, what is the beast? Imagemagick? Openjpeg? I want to fix it. Seems it makes a difference that you use windows (guess from your output) and I linux.

Yes, you do get a difference in images. I don't know why, but probably a difference between the builds, perhaps a problem in the JP2 library. I'm using the pre-built Windows binary v6.9.2-5. "convert -list format" shows:
It also works corectly for me with a Cygwin build of IM v6.9.3-7. Strangely, "convert -list format" desn't show JP2.

Dear snibgo,
indeed, just installed the IM for win64 and got identical jp2 from Bliss.bmp as well identical reverse bmp (i.e. Bliss.bmp->jp2(quality 100)->bmp). This simply confirms that something is weird in the linux implementation.
BTW, thanks for pointing out "compare -metric", was not aware about it.

SO, guys, what is responsible for this?

I'm ready to chip in and try to improve, but improve what? Confused ...

For Linux, I guess you built IM yourself? Then config.log should tell you about the libraries it used. Perhaps the library is old and buggy, or something, and should be updated, so you can re-build IM.

So, I guess, at least the problem is slightly rectified:
it is a glitch in conversion TO jpeg2000 which happens on linux. Now, What I could check so far is that:
1. This happen on my machine which is x64 linux gentoo with IM 6.9.0-3 Q16 x86_64 2016-04-28 and openjpeg 2.1.0
2. Things works as expected in win7pro x64 with IM 7.0.1 (downloaded and installed today)
3. What is the most embarrassing: things work normally on Ubuntu 10.04 (outdated) with IM 6.5.7 and openjpeg 1.3 (both outdated).

EDIT: I detected that a direct invocation of opj_compress (which defaults to lossless) generates a jp2 file IDENTICAL to original bmp as far as bitmaps are compared. I therefore strongly presume that it is IM, who calls opj_compress incorrectly when "-quality 100" is specified. I'm curious, why is it (my) linux specific???

I'm looking for a knowledgeable person (developer perhaps) to resolve this puzzle. As said before, I'm ready to participate.

Finally I confirm that the glitch of data loss when converting to jpeg2000 is specific to (at least mine, I mean got from gentoo repository) IM 6.9.0

Just compiled and installed IM 7.0.1 by hands and things work perfectly.

This implies that those who use IM 6.9.0 for linux must upgrade if jpeg2000 is the concern.