ImageMagick memory consumption

[dkode]

Configuration :
OS: CentOs

identify -version

Version: ImageMagick 7.0.6-7 Q16 x86_64 2017-08-12 http://www.imagemagick.org
Copyright: © 1999-2017 ImageMagick Studio LLC
License: http://www.imagemagick.org/script/license.php
Features: Cipher DPC HDRI Modules OpenMP
Delegates (built-in): bzlib cairo djvu fftw fontconfig fpx freetype gslib jng jpeg lcms ltdl lzma pangocairo png ps tiff webp wmf x xml zlib


identify -list resource
Resource limits:
Width: 107.374MP
Height: 107.374MP
Area: 33.3722GP
Memory: 15.5401GiB
Map: 31.0803GiB
Disk: unlimited
File: 48750
Thread: 4
Throttle: 0
Time: unlimited

____________________________________________________________________________________________________________________________

I wanted to know how to know the total memory, map and disk used by an imagemagick command.

I tried few options like -debug cache

Can anybody tell me how to exactly interpret the output?

Also I tried valgrind <command> to know the total heap used.

Please guide in this respect as i need to figure out sane resource limits for my application.

Also are the resource limits on per command basis ?

[fmw42]

Resource limits are set in your policy.xml file. There are Environmental variable that can be set per session. See https://www.imagemagick.org/script/reso ... nvironment. I do not know the you can change resources per command line. One of the IM developers or users more expert than I would have to respond.

[dkode]

Hi Fred,

I know how to set resource limits, my query is how to get the exact amount of memory used by a command as i am not able to fully comprehend debug cache's output. I need to reach a conclusion for setting resource limits but since i am not fully confident in getting resource consumption metrics for a command, i need some help in this area.

[fmw42]

I understand. You will need a response from one of the developers or some one more knowledgeable that me. Sorry I cannot help.

[fmw42]

Have you tried -debug resource? See https://www.imagemagick.org/script/comm ... .php#debug. I don't know if that helps or not. I have to used it before.

[snibgo]

I tried few options like -debug cache. Can anybody tell me how to exactly interpret the output?

The question is vague. The output might contain:

Code: Select all

2018-05-13T17:53:41+01:00 0:00.186 0.172u 7.0.7 Cache magick[3916]: cache.c/OpenPixelCache/3582/Cache

  open pink[0] (Heap Memory, 1000x1000x4 32000000B)

This says at a certain time the function OpenPixelCache is allocating heap memory for the image pink[0], for one million pixels each of 4 channels, and it needs 32 million bytes.

What else needs interpreting?

IM needs memory for pixels, and other structures. For small numbers of large images we might ignore "other structures" and just consider pixels. Memory needed per pixel depends on v6 or v7, and number of channels, and HDRI or not, and Q-number.

Your operating system will have facilities that tell you total memory usage per process, eg the Windows "tasklist" command. My process module "shell" enables us to run those commands at any stage during the execution of an IM command.

Also are the resource limits on per command basis ?

I'm unsure what you mean. Limits can be set per command. If you have two or more IM commands running simultaneously, the limits apply to the commands individually, not collectively. An IM process has no knowledge about any other IM processes that might also be running.

[dkode]

Hi Snibgo,

I tried -debug resource and got following output

2018-05-14T19:47:10+05:00 0:00.210 0.010u 7.0.6 Resource convert[4343]: resource.c/AcquireMagickResource/319/Resource
Width: 600B/0P/107.374MP
2018-05-14T19:47:10+05:00 0:00.210 0.010u 7.0.6 Resource convert[4343]: resource.c/AcquireMagickResource/319/Resource
Height: 457B/0P/107.374MP
2018-05-14T19:47:10+05:00 0:00.210 0.010u 7.0.6 Resource convert[4343]: resource.c/AcquireMagickResource/319/Resource
Area: 3290400B/3290400P/17.1799GP
2018-05-14T19:47:10+05:00 0:00.210 0.010u 7.0.6 Resource convert[4343]: resource.c/AcquireMagickResource/319/Resource
Memory: 3290400B/3290400B/8GiB
2018-05-14T19:47:10+05:00 0:00.270 0.020u 7.0.6 Resource convert[4343]: resource.c/AcquireMagickResource/319/Resource
Memory: 822600B/4113000B/8GiB
2018-05-14T19:47:10+05:00 0:00.270 0.020u 7.0.6 Resource convert[4343]: resource.c/RelinquishMagickResource/968/Resource
Memory: 822600B/3290400B/8GiB
2018-05-14T19:47:10+05:00 0:00.300 0.040u 7.0.6 Resource convert[4343]: resource.c/RelinquishMagickResource/968/Resource
Memory: 3290400B/0B/8GiB

Can you tell the interpretation of the line?

Memory: 822600B/3290400B/8GiB
Last part is RESOURCE_LIMIT. what about other two.

8GB is the MEMORY_LIMIT, one of them is memory used but what is other one? Please tell if you know.

[dkode]

I got the answer for above question myself by looking at MagickCore code https://www.imagemagick.org/api/MagickC ... ource.html .

The format is <Memory in the current step>/<Total memory used>/ MEMORY_LIMIT

Example:
total acquired=OB
Max limit=8GiB ( to check run : identify -list resource )

2018-05-14T19:47:10+05:00 0:00.210 0.010u 7.0.6 Resource convert[4343]: resource.c/AcquireMagickResource/319/Resource
Memory: 3290400B/3290400B/8GiB
---Acquired memory of size 3290400B, also total acquired=0B+3290400B=3290400B ---

2018-05-14T19:47:10+05:00 0:00.270 0.020u 7.0.6 Resource convert[4343]: resource.c/AcquireMagickResource/319/Resource
Memory: 822600B/4113000B/8GiB
---Acquired memory of size 822600B, total acquired=3290400B+822600B=4113000B ---

2018-05-14T19:47:10+05:00 0:00.270 0.020u 7.0.6 Resource convert[4343]: resource.c/RelinquishMagickResource/968/Resource
Memory: 822600B/3290400B/8GiB
---Released memory of size 822600B, total acquired=4113000B-822600B=3290400B---

2018-05-14T19:47:10+05:00 0:00.300 0.040u 7.0.6 Resource convert[4343]: resource.c/RelinquishMagickResource/968/Resource
Memory: 3290400B/0B/8GiB
---Released memory of size 3290400B, total acquired=3290400B-3290400B=0B---

So at the end all memory taken up has been released to the os.