Understanding ImageMagick's memory management

[matthias-ronge]

I have been rerading about the -limit argument, and about the corrsponding environment variables. However, I do not fully understand the meaning of the individual values.

I want to create a thumbnail of a 1.6 GB uncompressed TIFF file. I was told that the process takes four times the TIFF's size of RAM. I do not personally work with large files, but I am implemeting a web server program which uses ImageMagick, and I want my program to prevent this behaviour if a customer uses it with so large files. Therefore I want to understand better what the memory options mean, to set them properly.

-limit memory 150MB
The first part is clear: If the source TIFF is larger than 150 MB, it will not be loaded into memory but "memory mapped files" is used.

What exactly does this mean? I read the Wikipedia article on it but I do not understand it fully. Especially, what does that mean for the memory footprint? Assuming a page of memory is 4096 bytes on an average linux system. Does this mean that

• the memory used to load a 1.6 GB image is 1.6G / 4K = 410K of pointers to memory pages on disk? (3,2 MB if a pointer is 8 bytes) Which means to other applications on the same system, 3.2M of memory less is available? If this or something related is true, what does the -limit map value limit: The file size which disallows using the memory mapped file model, or the ram used for storing these pointers?
• The image is something like mounted as some kind of additional swap drive, not affecting the RAM available to other processes? (If this would be the case, I do not undestand why there is a map limit, and another disk option at all.)
• The image is read right from disk (does not need time to "read" it fully at the beginning of its processing), but will still make 1.6 GB of RAM unavailable to other processes, because 1.6 G of memory addresses point to the file?

-limit area 150MB
Seems clear: If the source TIFF is larger than 150 MB, the command will crash.

-limit disk 150MB
Seems clear: If the source TIFF is larger than 150 MB, the command will crash. Is there a difference to the -limit area? If so, which?

[magick]

An area limit forces the pixel cache to disk if the image dimensions exceed the limit.

Memory mapping increases I/O performance of a pixel cache on disk. See https://en.wikipedia.org/wiki/Memory-mapped_file.

Have you reviewed http://www.imagemagick.org/script/security-policy.php? You can set your resource limits in your security policy.

[snibgo]

I was told that the process takes four times the TIFF's size of RAM.

It isn't that simple. IM may take more or less than the uncompressed TIFF filesize. Memory needed per pixel depends on v6 or v7, and number of channels, and HDRI or not, and Q-number. For example, v7 Q16 HDRI needs 4 bytes/channel/pixel, and Q32 HDRI needs 8 bytes/channel/pixel.

Especially, what does that mean for the memory footprint?

I don't know. And it may vary between Unix and Windows, and perhaps between flavours of Unix.

I'm not sure what "-limit map" controls. Whatever it is, it consumes far less memory than the full pixel cache, so "-limit map 0B" doesn't save much memory. But it's probably a good idea to set this to the same as "-limit memory".


The units of "-limit area" is pixels, not bytes. It is the image columns multiplied by image rows.