change PNG endian without a pipe (need -set endian LSB/MSB)

[thomas12]

some Flir thermal cameras save the images a 16 Bit PNG with a wrong byte order
( for further infos see this post http://www.eevblog.com/forum/testgear/f ... #msg348398 )

here is a 16 Bit sample PNG with a wrong byte order


I tested all variants of +/- endian to change the byte order and the only solution I found is this:

Code: Select all

convert t1.png gray:- | convert -depth 16 -endian msb -size 320x240 gray:- -auto-level t3.png

I look for a faster solution without a pipe for using the "Filename Percent Escapes" from Imagemagick http://www.imagemagick.org/Usage/files/#save_escapes
but this doesn't work

Code: Select all

convert t1.png +endian -endian LSB -normalize -evaluate multiply 1 -auto-level 3a.png

Does anyone have an idea how to change the endian without using a pipe?

... I now the slow fx operator
this works fine

Code: Select all

convert t1.png -fx "u/256+int(u*65536)%256)/256" -auto-level t3.png

-----------------------------------------------
background:
with a color lookup table we get a usual thermal image:

[fmw42]

I can reproduce this on my IM 6.8.8.2 Q16 Mac OSX. I would think that there should be an easier way to do this in IM.

The file has endian=undefined. So I tried include -set endian lsb, but that did not work either.

Code: Select all

convert flir-e4-thermal-imaging-camera-teardown.png -set endian lsb -endian msb -auto-level show:

Of course one could edit the file in something like EXIFTool before processing it in IM. Do you have this kind of corrupt PNG often?

I am going to transfer this to the Developers forum for their review.

[thomas12]

Of course one could edit the file in something like EXIFTool before processing it in IM.

I asked Phil (the coder of exiftool) this question a year ago and he wrote
http://u88.n24.queensu.ca/exiftool/foru ... l#msg23672

I just return the PNG as is ...(ExifTool doesn't do image manipulations -- that's for other software.)

Phil doesn't touch the PNG. This is a stupid error/bug from the Flir camera.

Do you have this kind of corrupt PNG often?.

yes, there are many users with a hacked Flir E4 thermal camera (change thermal resolution from 80x60 to 320x240 )
http://www.eevblog.com/forum/testgear/f ... -teardown/

[snibgo]

This seems to reverse bytes. There may be a better way. Windows syntax:

Code: Select all

convert ^
  in.png ^
  ( -clone 0 -evaluate RightShift 8 ) ^
  ( -clone 0 -evaluate And 255 -evaluate LeftShift 8 ) ^
  -delete 0 ^
  -compose Plus -composite ^
  out.png

[fmw42]

I thought perhaps one could just set the endian in Exiftool to lsb and then have IM change it. I think the problem is that IM sees it as undefined and I guess there is no -set endian at this time. That would certainly be the best solution.

I checked EXIFtool, but it does not seem to report the endianness

Code: Select all

exiftool -s -ee -g1 -u -n -D flir-e4-thermal-imaging-camera-teardown.png
---- ExifTool ----
    - ExifToolVersion                 : 8.71
---- System ----
    - FileName                        : flir-e4-thermal-imaging-camera-teardown.png
    - Directory                       : .
    - FileSize                        : 52671
    - FileModifyDate                  : 2014:01:16 10:43:25-08:00
    - FilePermissions                 : 644
---- File ----
    - FileType                        : PNG
    - MIMEType                        : image/png
---- PNG ----
    0 ImageWidth                      : 320
    4 ImageHeight                     : 240
    8 BitDepth                        : 16
    9 ColorType                       : 0
   10 Compression                     : 0
   11 Filter                          : 0
   12 Interlace                       : 0
---- Composite ----
    - ImageSize                       : 320x240

If it did, then one could just change the endian flag and then have IM change the data appropriately.

Any way, user snibgo has another solution, though I am not sure which would be faster, his or yours.

Perhaps PNG does not store endianness? The PNG developer would have to comment.

[thomas12]

@snibgo

a great solution, 11x faster as my fx operator hack

Code: Select all

$ time convert 1.png -fx 'u/256+int(u*65536)%256)/256' out1.png
real	0m0.876s

$ time convert 1.png \( -clone 0 -evaluate RightShift 8 \) \( -clone 0 -evaluate And 255 -evaluate LeftShift 8 \) -delete 0 -compose Plus -composite out.png
real	0m0.077s

$ time convert 1.png gray:- | convert -depth 16 -endian msb -size 320x240 gray:- out3.png
real	0m0.035s

thanks

-----------------------------------

Exiftool can't manipulate the PNG binary.
Phil (exiftool) says, it's ImageMagick part to correct the wrong byte sequence

this is a RAW dump from EXIF Header of the FLIR camera

$ exiftool -v3 FLIR0080.jpg | grep -A40 "FLIR Record 0x01"
| FLIR Record 0x01, offset 0x0f24, length 0xcddf
| RawData (SubDirectory) -->
| - Tag 0x0001 (52703 bytes):
| 0f24: 02 00 40 01 f0 00 00 00 00 00 00 00 3f 01 00 00 [..@.........?...]
| 0f34: ef 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 [................]
| 0f44: 89 50 4e 47 0d 0a 1a 0a 00 00 00 0d 49 48 44 52 [.PNG........IHDR]
| 0f54: 00 00 01 40 00 00 00 f0 10 00 00 00 00 04 d6 3e [...@...........>]
| 0f64: f4 00 00 20 00 49 44 41 54 78 01 5c dd db af 6d [... .IDATx.\...m]

exiftool export the PNG with same byte sequence

Code: Select all

$ hexdump -C flir-e4-thermal-imaging-camera-teardown.png | head
00000000  89 50 4e 47 0d 0a 1a 0a  00 00 00 0d 49 48 44 52  |.PNG........IHDR|
00000010  00 00 01 40 00 00 00 f0  10 00 00 00 00 04 d6 3e  |...@...........>|
00000020  f4 00 00 20 00 49 44 41  54 78 01 5c dd db af 6d  |... .IDATx.\...m|
00000030  5b 76 df f5 39 f7 5e fb  ec cb b9 94 ab ca 75 af  |[v..9.^.......u.|
00000040  54 ca ae 60 c7 c1 94 65  1b 63 43 12 94 c8 41 21  |T..`...e.cC...A!|

[snibgo]

"-fx" is usually slower than any other feasible solution. I notice your expression has unbalanced parentheses, though it doesn't seem to matter. I think it should be:

Code: Select all

u/256+(int(u*65536)%256)/256

[thomas12]

ups, you are right
but ImageMagick think for me

Code: Select all

$ echo -n ABCDEF | convert -size "3X1" -depth 16  gray:- \( -clone 0 -evaluate RightShift 8 \) \( -clone 0 -evaluate And 255 -evaluate LeftShift 8 \) -delete 0 -compose Plus -composite - | hexdump -C
00000000  42 41 44 43 46 45                                 |BADCFE|

$ echo -n ABCDEF | convert -size "3X1" -depth 16  gray:- -fx 'u/256+int(u*65536)%256)/256'  - | hexdump -C
00000000  42 41 44 43 46 45                                 |BADCFE|

[fmw42]

I was not saying that Exiftool should change the pixel data, only to change the meta data tag for endianness. But it does not look like it can even handle that at least not for PNG.

[thomas12]

its a job for a new PNGTool

Thanks for help!!

[fmw42]

I do not know enough about PNG. However, the PNG developer, glennrp, may be able to shed more light on this or even set an option for some kind of PNG chunk to specify the endianness.

[glennrp]

Perhaps PNG does not store endianness? The PNG developer would have to comment.

It does not. Only network byte order is allowed in PNG files. There is a libpng function for swapping bytes if
necessary: png_set_swap(). It wouldn't be hard to make the ImageMagick PNG decoder use it optionally.

[glennrp]

Checked in to IM6 SVN revision 14464 a new "-define png:swap-bytes" option.
Note that for the original problem, you'll still need to do -auto-level or something.

If the "-define png:swap-bytes" option is present, it must appear on the commandline
ahead of the input filename. Libpng will do the swapping operation during the
decoding process.

If this is satisfactory, I'll update IM7 as well.

[thomas12]

Checked in to IM6 SVN revision 14464 a new "-define png:swap-bytes" option.

a perfect solution for this problem
thanks

[glennrp]

a perfect solution for this problem
thanks

I've checked the new feature into IM7

Glenn