Memory Allocation Error

[matthew]

I have a number of TIF files that I am trying to convert to Group4 compressed TIF files. My processed worked fine on the smaller files but on the larger (~1GB) files I keep getting the following memory allocation error:

convert.exe: Memory allocation failed `f:\images\12015_52737_202124.TIF' @ error
/quantize.c/QuantizeImage/2726.

The easy answer is that I'm just running out of memory but I'm running this on a machine with 32GB of memory and when I monitor the memory during execution it never seems to request more than a few GB. It also uses quite a bit of disk space but I have over 300GB free on the partition for the temporary files and it does not seem to be filling up.

Code: Select all

Version: ImageMagick 6.8.0-4 2012-10-30 Q16 http://www.imagemagick.org
Copyright: Copyright (C) 1999-2012 ImageMagick Studio LLC
Features: OpenMP

Code: Select all

File         Area       Memory          Map         Disk    Thread         Time
-------------------------------------------------------------------------------
1536     34.355GB    15.998GiB    31.996GiB    unlimited         8    unlimited

I've read http://www.imagemagick.org/Usage/files/#massive and have tried playing with the limits but I haven't had any luck.

Any help or insight would be much appreciated.

[magick]

If you can grab ImageMagick 6.8.1-8, the latest release, that may help. There are some patches that address large memory / files. Now try this command:

• convert -limit memory 0 -limit map 0 image.tif -compress group4 new.tif

If that fails, let us know and we have a couple more recommendations.

[matthew]

-limit memory 0 -limit map 0

This seems to have helped, however now some of them are causing me to run out of disk space.

[magick]

Set your temporary disk path to a partition with plenty of free space with the MAGICK_TMPDIR environment variable or the -define registry:temporary-path=/data/tmp command line option.

[matthew]

How much disk space is "plenty"? It filled up a partition with 300+ GB free.

[magick]

The pixel cache grows as required by the image or image sequence. For 300 GB you would need a 50000 by 50000 pixel image or larger or perhaps you have an image sequence with modest size frames but hundreds of them. Or perhaps you have a corrupt image that incorrectly returns a huge image size. Post a URL to your image and we will download it and try to reproduce the problem.

[matthew]

I am not able to post the images because they contain sensitive information. They vary but as an example one is 1764x2248 grayscale with 179 pages; another is 1721x2216 grayscale with 441 pages. It may partly be because it doesn't always clear the cache after it finishes processing a given image.

[magick]

ImageMagick opens each of the frames in an image sequence. With that many frames, 300GB of disk seems reasonable. If that's a problem you can read just a frame or two at a time, e.g. convert 'image.tif[3-5]'.