Appending Large Images Using Streams

[kevinswarner]

I am working on a project where I need to append two very large TIF images vertically. Each image is several hundred MBs. We have very limited resources as we are using Amazon Lambda function compute service. So, we have memory and disk limitations.

I have been using the following to do the work.

Code: Select all

magick convert image1.tif image2.tif -append result.tif

Because of the size of the images, we are running into resource problems, both disk and memory.

I have tried using -limit, but this only partially helps.

I would like to know if I can use the "stream" option (https://www.imagemagick.org/script/stream.php) in ImageMagick to perform this operation using less memory and disk space. I understand the basic concepts of streaming, but from all the examples I have seen on using "stream", I cannot figure out how to do this. Again, just looking for example usage.

Thanks for your help.

[fmw42]

I cannot answer your question about streaming. But in Imagemagick 7, you should use just magick and not magick convert, nor just convert.

[magick]

Streaming won't help. Have you reviewed https://www.imagemagick.org/script/arch ... tera-pixel? You should be able to convert any sized image as long as you point the pixel cache to a disk partition that has plenty of free space. To track resource requirements, add -debug cache to your command-line.

[kevinswarner]

Thanks, but we are also limited on disk space. Our limits are 1 GB temporary disk space and 3GB memory. The two files we are appending are around 300 MB each. Even with -limits in place, we are exceeding one resource or the other. What is the reason streaming will not work? I guess I am just looking for more detail.

I am moving to using a custom build of IM with quantum depth of 8 to see if that helps. I have read in a few places that this will "halve" the memory requirements / usage.

Thanks.