Enormous RAM consumption while concatenating TIFFs

[ZYV]

Hi Magicians!

This is not a bug, strictly speaking, because anyhow IM does its job, but somehow, I find that this might render ImageMagick unusable in many practical cases. I need to concatenate few thousands TIFF grayscale frames in one multipage TIFF image. For this I run the following command:

convert *.TIF -adjoin bleaching.tiff

For 3517 files (361952 bytes each) it only takes 7 minutes which is perfectly fine with me and results in a 1361M document. However, the amount of RAM it consumes in the process is around 12.1 Gb. Luckily I have a machine with 32 Gb of RAM available, but before I realized this I already almost killed my laptop from overheating (Linux ran out of RAM and swap and ended up killing itself after reaching a critical temperature of 90 degrees).

Is there anything I am doing wrong? Why such a high amount of RAM is required for seemingly trivial operation? Is there a workaround? Is this going to be considered as a bug and fixed later on or I will need to buy even more RAM to concatenate larger movies?

Thanks!

zaytsev@loki:~$ uname -a
Linux loki 2.6.28-18-server #60-Ubuntu SMP Fri Mar 12 05:11:07 UTC 2010 x86_64 GNU/Linux

zaytsev@loki:~$ convert -version
Version: ImageMagick 6.4.5 2009-06-04 Q16 OpenMP http://www.imagemagick.org
Copyright: Copyright (C) 1999-2008 ImageMagick Studio LLC

[magick]

Add -limit memory 1 -limit map 1 to your command line to force the pixel cache to disk instead of memory. You might want to install the latest ImageMagick release as well.

[ZYV]

Hi!

While your "disk cache" solution does work, it's terribly slow. I will consider it when I run out of memory on the big machine though.

I have compiled the latest version from sources (I only need to convert 8-bit grayscale images):

Code: Select all

zaytsev@loki:~$ convert -version
Version: ImageMagick 6.6.1-3 2010-04-15 Q8 http://www.imagemagick.org
Copyright: Copyright (C) 1999-2010 ImageMagick Studio LLC
Features: OpenMP

It works way much faster and uses like two times less memory which corresponds to my expectations, provided that you keep all the files in memory at the same time.

However, I find this kind of behavior rather weird. Why isn't it only keeping the current TIFF page in memory and flushing the stream to the disk directly? Is there any chance that such behavior is going to be implemented in future versions?

Thanks!

[magick]

Read http://www.imagemagick.org/script/architecture.php. It describes in great detail our design decisions in regards to the pixel cache.

[ZYV]

Thanks for the concise replies!