Stream zero output on palette (and grayscale) PNG

[abukaj]

When I am trying to stream file Ambermoon-lyramion.6400x6400.png my output is a stream of zeros.

Code: Select all

$ stream Ambermoon-lyramion.6400x6400.png - | hexdump
0000000 0000 0000 0000 0000 0000 0000 0000 0000
*
7530000

However other IM tools seem to open it properly (display, convert etc.).

I think the problem is connected with palette mode of the PNG.

Code: Select all

$ identify Ambermoon-lyramion.6400x6400.png            
Ambermoon-lyramion.6400x6400.png PNG 6400x6400 6400x6400+0+0 8-bit PseudoClass 256c 6.562MB 0.000u 0:00.009

However when the image is converted to GIF - then stream streams it correctly, so it is not general palette mode issue.

My version of IM is:

Code: Select all

$ stream -version
Version: ImageMagick 6.6.9-7 2014-03-06 Q16 http://www.imagemagick.org
Copyright: Copyright (C) 1999-2011 ImageMagick Studio LLC
Features: OpenMP

The file is available at: https://dl.dropboxusercontent.com/u/223 ... 0x6400.png

I know I can convert image and then stream it, but I need a lightweight tool for automated streaming of large images and convert is not a lightweight tool.

[magick]

We can reproduce this problem. If you first convert the PNG to PNM, streaming works. We'll investigate the problem and hopefully have a patch soon.

[abukaj]

Thanks for the piece of advice, however what I need is to stream the image without loading it into the memory (AFAIK convert does it that way).

I am developing a server for hosting images of brain tissue of high resolution, so the solution should be also format-independent. Luckily it is very unlikely to have a palette PNG uploaded by our users.

[glennrp]

We understand your requirement, but I don't think the fix is trivial. PNG is designed to be streamable, and so is libpng, but applications must use a different part of libpng (the progressive reader, found in pngpread.c, instead of the sequential reader, found in pngread.c), to achieve streaming of a PNG datastream. By streamable, I mean that the encoder can receive a buffer containing only a part of the image, process and emit the buffer, and wait for another buffer. Note that in this context, the term "progressive" has nothing to do with PNG interlacing.

There are demonstrations of progressive reading in the libpng distribution. "example.c" is a commented example that does not actually run, and "contrib/gregbook" contains "rpng*.c" examples that do work. I think the licensing is OK (dual-licensed GPLV2 and BSD-like).

Certainly there is a bug that is causing "convert" to write zeroes instead of the correct image pixels, though.

[glennrp]

If you first convert the PNG to PNM, streaming works.

By this I assume you mean that "convert" is able to stream in the PNM, not that the conversion from PNG to PNM works in a stream.

[magick]

Right, lots of other image formats stream correctly, however, PNG does not. Depending on how the coder handles the image format, streaming may not be possible for certain image formats.

[glennrp]

Right, lots of other image formats stream correctly, however, PNG does not. Depending on how the coder handles the image format, streaming may not be possible for certain image formats.

It's not the PNG format that is the problem; it's the fact that we are using the sequential decoder in libpng instead of the progressive decoder.

[abukaj]

I found that the same problem is with grayscale-PNGs.

applications must use a different part of libpng (the progressive reader, found in pngpread.c, instead of the sequential reader, found in pngread.c), to achieve streaming of a PNG datastream.

Would it not move the memory usage of stream out of control? However at the moment it seems that in case of interlaced PNG stream reads whole image into memory. Same is with grayscale non-interlaced PNG.

Possibly use of png_set_palette_to_rgb and png_set_gray_to_rgb functions of libpng would be a kind of a walkaround?

Certainly there is a bug that is causing "convert" to write zeroes instead of the correct image pixels, though.

Convert works for me with that image.

Right, lots of other image formats stream correctly, however, PNG does not. Depending on how the coder handles the image format, streaming may not be possible for certain image formats.

How can I simply check, if the image is suitable for lightweight streaming it with stream? By "lightweight streaming" I mean not loading a whole image into memory before the RGB stream can be generated.