resize / sample a very large image - out of memory / ram

[jggretton]

Hello all, I've searched around for a while but haven't yet found anything conclusively answering my question. If you know of a post /site where this has been answered already please do just send me there.

I'm trying to resize very large images (uploaded to a website by professional photographers with crazily high megapixel cameras) but my server can only handle images up to about 10,000x10,000 pixels. Ideally I need to convert upto 20,000x20,000px images into small thumbnails.

I'm running ImageMagick on the command line through PHP ( using exec() ).

I've found that using -sample instead of -resize is much faster, but still seems to use just as much RAM.

Does anyone know of a way to resize images with better memory management in ImageMagick?

I might be able to install extras on my server, but it'll be a bit of a pain so would prefer a clean solution if it was possible.

It's not so relevant to the ImageMagick forum but if it's not going to be possible, does anyone know of any web services that do this with a nice API?

Many thanks for any help you can provide,

James

[fmw42]

You are probably running out of RAM. see http://www.imagemagick.org/Usage/files/#massive

[jggretton]

Great, thanks for the link. I've had a look through and I think stream might be useful - it will let me extract chunks of the image to resize piece by piece and re-combine at the end. Fairly long winded, but I think it'll work even when both ram and disk space are low.

I'm a little surprised that there's not a more automated way of doing this as I think I'll probably end up running a number of stream/convert commands which doesn't sound very efficient, but hey, I'm going to give it a go and see what happens!

I'm also slightly worried about what file types stream does and doesn't work with, if anyone knows of a conclusive list i'd be grateful.

Again, thanks for the help,

James

[anthony]

Great, thanks for the link. ... Fairly long winded

Care to re-write and mail it to me

I'm a little surprised that there's not a more automated way of doing this as I think I'll probably end up running a number of stream/convert commands which doesn't sound very efficient, but hey, I'm going to give it a go and see what happens!

The only reason it is not automated is, no one has provided scripts to do so.

Also I have had information on extracting a piece from a larger image, but very little in the way of adding a piece back into a larger image. It should be possible.. for example use stream to get the first part, the row where the piece is merged, and the bottom half, and then feed it into a 'write back to new image format'.

Code: Select all

I'm also slightly worried about what file types stream does and doesn't work with, if anyone knows of a conclusive list i'd be grateful.

I wouldn't mind knowing that too. But depends on the file format and the coder. Also will not work on 'interlaced' sub-formats of various image file formats.

Hmmm it may be that NetPBM (also known as PbmPlus) can stream images row by row when IM coder doesn't. That image processing package is designed to 'streaming' in mind, but not all utilities in the package does 'streaming'. that is some always reads whole images into memory (caching).

We really need someone who has an interest in this area to look at all the streaming techniques and write up there findings. I'll even host it if you follow my Web page style. Lens correction, and Windows pages are examples of pages I did not write, but I host.