IM and out-of-memory on debian

[Alexis Rombaut]

According to the documentation IM should automatically switch to a file base cache when memory becomes scarce. What I however experience is that it is killed by OS when memory runs out instead of automatically switching to a file based pixel cache. Is there a way for IM do switch to file based cache without using the -limit option, namely by the automatic system described in the documentation.

IM documentation

This automatically happens when memory starts to become a problem, but you can also force it using the -limit option. It is however slower (very much slower)

To illustrate how ImageMagick utilizes resource limits, consider a typical image resource request. First, ImageMagick tries to allocate the pixels in memory. The request might be denied if the resource request exceeds the memory limit or if the system does not honor the request. If a memory request is not honored, the pixels are allocated to disk and the file is memory-mapped. However, if the allocation request exceeds the map limit, the resource allocation goes to disk. In all cases, if the resource request exceeds the area limit, the pixels are automagically cached to disk. If the disk has a hard limit, the program fails.

It appears instead that IM is being killed by the kernel OOM killer

grep -i kill /var/log/messages*
/var/log/messages:Dec 12 14:00:54 ... kernel: [1114962.091332] convert invoked oom-killer: gfp_mask=0x201da, order=0, oom_adj=0, oom_score_adj=0
/var/log/messages:Dec 12 14:00:54 ... kernel: [1114962.091465] [<ffffffff810b74e7>] ? oom_kill_process+0x49/0x271

From malloc(3)

By default, Linux follows an optimistic memory allocation strategy.
This means that when malloc() returns non-NULL there is no guarantee
that the memory really is available. In case it turns out that the
system is out of memory, one or more processes will be killed by the
OOM killer. For more information, see the description of
/proc/sys/vm/overcommit_memory and /proc/sys/vm/oom_adj in proc(5),
and the Linux kernel source file Documentation/vm/overcommit-
accounting.

[magick]

Either turn off the optimistic memory allocation strategy in your OS or use conservative limits on your ImageMagick command-line or API calls such as:

• convert -limit memory 2GiB -limit map 2GiB image.jpg image.png

You can exempt the ImageMagick process from the OOM killer by setting its priority to -17.