Legacy ImageMagick Discussions Archive

I am using ImageMagick in a Linux application as part of an image translation pipeline. I've been having problems with the server OOM killing processes. The command I am running can be replicated from the command line:

$ convert -resize x64 s1.svg s1.gif

s1.svg looks like this:
The command when run consumes 128Mb (134221824 bytes) of memory (as seen through strace):
I traced the memory utilisation to the following lines in "GetCubeInfo()" in MagickCore/quantize.c:
In the above code, which appears to be independent of the size of the file being processed, length evaluated to 2^24 and size is 8, so the call ends up requesting 2^27 bytes (128Mb) of memory.

The purpose of the application is to convert two differing SVGs into GIFs and then stitch them together as an animated GIF for web display. The problem is that the memory being used by this process is causing the OS to OOM kill processes due to the memory requirement when multiple converts run simultaneously.

Is there any reason why GIF needs such a huge amount of memory for what works out to be a 5k GIF file?

Surely the dither resources (the memory allocation in GetCubeInfo) should be related to the size of the source file?

Setting resource limitations either in the call or via shell environment doesn't help. They are ignored.

Environment: debian Jessie (3.16.39-1+deb8u2)
ImageMagick: v7.0.6 (compiled and run on host)

I've also tried this on RedHat using packages and got the same result.

What SVG renderer are you using? Imagemagicks MSVG/XML, RSVG delegate or Inkscape? You can check via and see which is listed for the line with SVG. Note that XML will be the worst, RSVG next and Inkscape the best.

Your IM 6 command should be
You may want to add -depth 8 or less and +dither -colors XX to reduce the file size. You may also want to use -density YY before s1.svg rather than -resize to reduce the resulting pixel dimension. That may be faster.

Imagemagick will rasterize your vector SVG file, so that is why the file size gets large.

On my IM 6.9.9.5 Q16 Mac OSX, I get the following:
real 0m0.036s
user 0m0.036s
sys 0m0.012s
real 0m0.065s
user 0m0.122s
sys 0m0.016s

Using IM 7.0.6.5 Q16 HDRI, I get:
real 0m0.115s
user 0m0.180s
sys 0m0.031s


So this is not causing any problems. My SVG renderer is

SVG SVG rw+ Scalable Vector Graphics (RSVG 2.40.17)

Sorry, I do not know how to look at the memory usage.

P.S. Do you have restricted memory allocation in your policy.xml file?

Thank you for your quick reply

My SVG renderer is
The SVG files created by the application are quite small, so although they process badly, they don't cripple the system.

When I re-ran with the arguments re-arranged:
It has no effect on memory usage. Still using 128Mb of memory.

However, when I re-ran with the +dither option, the memory usage drops to 8Mb, which is more than acceptable
This looks like it solves my problem.

I also note that
uses 128Mb of memory
uses only 8Mb

So PNG doesn't appear to use the same buffer by default.

Dithering appears to be what fixes the memory usage

>> P.S. Do you have restricted memory allocation in your policy.xml file?

No, I've not installed any special restrictions. The policy file is what was installed by default.

I'll see if I can rebuild the same command with the full pipeline using the +dither option

Thank you

Try installing Inkscape. IM will use that if it is installed. It may render better that IM's MSVG/XML. But then it just could be as you suggest in the gif writer.

I suspect the problem is independent of SVG. You can test this: I suspect the memory problem is entirely in the second convert.

The image, tmp.png, is 124x30 pixels, entirely black. Is this what you get? Being only one colour, I would hope a colour-reduction process would perform quickly, without much memory. Perhaps something is going wrong.

A developer might take a look.

Meanwhile, "-treedepth" might reduce memory usage. See http://www.imagemagick.org/script/comma ... #treedepth

ImageMagick 7.0.6-6, when released, permits large memory allocations for certain algorithms on disk rather than heap. Simply set this policy in policy.xml:
The trade off is reduced memory with a slight degradation in performance.

snibgo wrote: ↑2017-08-10T04:21:41-07:00 I suspect the problem is independent of SVG
I suspect the memory problem is entirely in the second convert.
Meanwhile, "-treedepth" might reduce memory usage

If a GIF dither occurs, it appears the code pointed out at the top will allocate a 128Mb buffer for this purpose. The code doesn't make an allowance for the size or content of the inbound graphic from what I can see, so it doesn't matter what the source graphic is.

I tried this with every value from 0 to 8 to see how it affects the performance with the following command:
In all cases it still required 128Mb to execute

The magic seems to come with disabling dither which prevents the dither buffer from being allocated (many thanks to fmw42 for pointing this out)

magick wrote: ↑2017-08-10T05:50:30-07:00 ImageMagick 7.0.6-6, when released, permits large memory allocations for certain algorithms on disk rather than heap

This sounds like it could help as it wouldn't OOM my system, but my point would still be why the 128Mb is required in the first place? Shouldn't the dither buffer be based on the source image?

We allocate a cache to improve performance for color lookup. Its size is based on the RGB triple whose range is [0..255]. It is not based on image size. We could have an option to not use the cache and lookup every color by traversing the color cube, but that would slow down processing considerably. Performance improves considerably with our current solution of allocating the cache to disk and memory mapping it-- if you request it with the max-memory-request security policy. There are algorithms that perform color reduction with a smaller memory footprint, however, our original test images were computer generated spheres. The current algorithm was deemed visually optimal for these types of images.