Page 1 of 1

6.6.4-2-Q16 montage memory leak

Posted: 2010-09-16T09:45:20-07:00
by choyj
I setup a new server moving from XP/32-bit with Imagemagick 6.5.3-Q16 to a new box with Windows 7 x64 and recently installed 6.6.4-0-Q16(static) and tried to run the same montage.exe command I once ran on my XP box and now I see montage.exe consuming memory at about almost 1GB per 4 seconds. I figured 6.6.4-2-Q16 (static) might fix it, but the leak still happens. VC++ 2010 Redistributable was also installed.

Here's what I tried to run:

Microsoft Windows [Version 6.1.7600]
Copyright (c) 2009 Microsoft Corporation. All rights reserved.

C:\Users\choyj>cd "\Program Files\ImageMagick-6.6.4-Q16"

C:\Program Files\ImageMagick-6.6.4-Q16>montage.exe -tile 10 -geometry 192x270 -size 192x270 "D:/aa/*" temp.jpg
<montage.exe starts sucking up memory till Windows becomes sluggish and unusable, so I break out>
^C

C:\Program Files\ImageMagick-6.6.4-Q16>dir /w d:\aa
Volume in drive D is BIG D
Volume Serial Number is 880F-3C31

Directory of d:\aa

[.] [..] 0001.JPG 0002.JPG 0003.JPG 0004.JPG 0005.JPG
0006.JPG 0007.JPG 0008.JPG 0009.JPG 0010.JPG 0011.JPG 0012.JPG
0013.JPG 0014.JPG 0015.JPG 0016.JPG 0017.JPG 0018.JPG 0019.JPG
0020.JPG 0021.JPG 0022.JPG 0023.JPG 0024.JPG 0025.JPG 0026.JPG
0027.JPG 0028.JPG 0029.JPG 0030.JPG 0031.JPG 0032.JPG 0033.JPG
0034.JPG 0035.JPG 0036.JPG 0037.JPG 0038.JPG 0039.JPG 0040.JPG
0041.JPG 0042.JPG 0043.JPG 0044.JPG 0045.JPG 0046.JPG 0047.JPG
0048.JPG 0049.JPG 0050.JPG 0051.JPG 0052.JPG 0053.JPG 0054.JPG
0055.JPG 0056.JPG 0057.JPG 0058.JPG 0059.JPG 0060.JPG 0061.JPG
0062.JPG 0063.JPG 0064.JPG 0065.JPG 0066.JPG 0067.JPG 0068.JPG
0069.JPG 0070.JPG 0071.JPG 0072.JPG 0073.JPG 0074.JPG 0075.JPG
0076.JPG 0077.JPG 0078.JPG
78 File(s) 124,617,063 bytes
2 Dir(s) 2,039,721,279,488 bytes free

Am I doing something wrong or is there truly a bug?

Re: 6.6.4-2-Q16 montage memory leak

Posted: 2010-09-16T11:39:17-07:00
by magick
Add -limit area 0 to your command line. Its slow but it uses disk rather than memory to create your montage.

Re: 6.6.4-2-Q16 montage memory leak

Posted: 2010-09-16T12:49:10-07:00
by choyj
I'm curious to know why it takes so much memory. I never had this problem on XP with 6.5.3-6-Q16. Is it because I was using 32-bit and now that I'm using 64-bit, it'll use more than 2X the amount of memory needed? Or maybe there was a huge change from 6.5 to 6.6 to use more memory?

I did try what you suggested and it was too slow. So I cut down the number of files to just 10 files (average size of about 1.5MB per jpg file) and setup a tmp directory to see what was happening:

montage.exe -define registry:temporary-path=D:/data -limit area 0 -tile 10 -geometry 192x270 -size 192x270 "D:/aa/*" test.jpg

Looking at the D:/data directory, I notice it creates 10 magick-xxxxxxxx files of about 250MB each consuming a total of 2.5GB disk space for just 10 files! Then it appears a bunch of threads got launched to take each of the 250MB magick-xxxxxxxx files and convert them into approximately 400K files each while deleting the 250MB files as each one was finished processing. And then those 400K files got translated into the final output: a 143K test.jpg file.

I have yet to see if I see similar results with the same set of 10 files on the older XP machine, but I'm going to try again tonight and see what happens because that just seems crazy:

10 files for a total of 15MB needing 2.5 GB of memory (>150X the amount of memory need for each file) to create a small 143K montage file! That explains the huge consumption in memory.

Is there an option to optimize the way montage does this type of work? i.e. take one 1.5MB jpg file from the queue, create that huge 250MB raw file and after that, take that 250MB raw file and convert it to a 400K file before moving onto the next file? Is that possible? That would be much more efficient in terms of memory usage than to stuff all those huge raws into the cache.

Keep in mind that I'm not familiar with the inner workings of how montage.exe works. All the threading stuff I mentioned above is just me guessing what's going on behind the scenes.

Re: 6.6.4-2-Q16 montage memory leak

Posted: 2010-09-16T14:15:03-07:00
by magick
Don't use -size. Instead use -define jpeg:size=192x270.

Re: 6.6.4-2-Q16 montage memory leak

Posted: 2010-09-16T14:32:43-07:00
by choyj
That works so much better! I guess some options might have changed. Good to know and Thanks!