Page 1 of 1
Memory Leak
Posted: 2015-04-01T12:27:47-07:00
by Yay295
Using the Windows binary release ImageMagick-6.9.1-0-Q16-x64-dll.exe on a 64bit Windows 7 with 16GB of RAM.
ImageMagick Code Used
Code: Select all
convert big/frame%05d.png[1-3909] big/frame%05d.png[6067-34405] -resize 240 +dither -fuzz 1% -layers OptimizePlus -layers OptimizeTransparency big.gif
The folder 'big' is about 60GB and contains 37,332 1080p png images. Don't ask why I'm doing this; I'm not sure myself.
Windows Task Manager showed convert.exe using only about 637MB, but 90% of my RAM was in use. After closing the cmd window to stop the operation, the amount of RAM in use slowly decreased to appropriate levels (16%). convert.exe didn't close (and couldn't be force-closed) until RAM levels were back to normal.
I know there is a command to limit the amount of RAM used, but I let this run overnight so I thought it would be fine.
Re: Memory Leak
Posted: 2015-04-01T12:40:15-07:00
by snibgo
Convert is reading about 32000 images of 1080x1920 pixels. At 8 bytes/pixel, this needs 531 GB of memory. Thus, disk will be used, and it will be slow.
I would do this by first looping through the 32000 images, resizing each one (to 240x427). I don't know what dither or fuzz settings do in your command. The layers operations need all the images, so this is merely 26 GB.
Re: Memory Leak
Posted: 2015-04-01T12:48:55-07:00
by Yay295
That's what I did first. I just wanted to try doing it this way instead. Also, you have your dimensions backwards. 1080p is 1920x1080, which would make my resized gif 240x135. All 37,332 frames at that size only take up 1.5GB, and compress to a 118MB gif. This doesn't change the fact that ImageMagick is leaking memory somewhere.
Re: Memory Leak
Posted: 2015-04-01T13:05:53-07:00
by snibgo
... you have your dimensions backwards ...
So I did. Sorry.
240*135*8*37000/1e9 = 9.6 GB. This is how much memory is needed to read the images.
You don't say if your commands are failing. If you kill a command, this might cause memory (and disk) leakage. Things will take time to get back to normal. How do you know there is memory leakage?
Re: Memory Leak
Posted: 2015-04-01T13:40:27-07:00
by Yay295
After taking a closer look, I don't think there is one. It looks like my monitoring program froze when it ran out of RAM, so it looked like ImageMagick was still using all of it. Everything looked fine in the Resource Monitor.
Re: Memory Leak
Posted: 2015-04-06T10:47:39-07:00
by glennrp
I think you should be using mogrify instead of convert for this project. convert reads in the whole batch of images, then writes them out. mogrify reads the images one at a time, read-write-read-write-etc., and only needs enough memory to process one image. So,
Code: Select all
mogrify -resize 240 -quality 20 -format PNG big/frame%05d.png[1-3909] big/frame%05d.png[6067-34405]
convert big/frame*.PNG +dither -fuzz 1% -layers OptimizePlus -layers OptimizeTransparency big.gif
The intermediate files are frame*.PNG so you can distinguish them from the input frame*.png
I suggest using a low quality value for speed, since these are temporary files that don't need highly optimized compression.