Too much memory usage

[naoliv]

Hi!

These two (http://bugs.debian.org/389604 and http://bugs.debian.org/349200) bugs on Debian are related about memory usage with ImageMagick.

The first one is about display using too much memory to display images.
Using this image, for example: http://naoliv.googlepages.com/image.png, with 1.7 MB of size.

Doing a simple "display image.png" I see it using 248 MB of RAM here, while when using feh, it uses only 53 MB.

The second problem (bug 349200) is also related. The user is using montage with some JPEG images (summing 140 MB on his first test and 188 MB on his last one).
Memory usage explodes when he does this:

Code: Select all

montage -label %f -background '#fff' -geometry +4+4 -resize 10% *-*.jpg index.jpg

Thank you!

[magick]

Add this to your command line to trade off memory usage for I/O usage: -limit memory 16mb -limit map 32mb. Another option is to build the 8-bit version of ImageMagick (add --with-quantum-depth=8 to the configure command line). The 8-bit version uses 100% less memory than the default 16-bit version.

[broucaries]

Hi,

User use this command as recommended:
$ montage -limit memory 16000000 -limit map 160000000 -label %f -background '#fff' -geometry +4+4 -resize 10% '*_*.jpg' index.jpg

But get
...
montage: unable to extend cache `20080216_150344.jpg': No space left on device.
montage: unable to extend cache `20080217_015805.jpg': No space left on device.
montage: unable to extend cache `20080217_015859.jpg': No space left on device.

top said
8247 flip 20 0 203m 156m 152m D 17.3 16.6 0:04.74 montage

And it does not work

Any test to do?

[magick]

The exception is telling you that there is not enough free space on your disk to create the image. Try freeing space on your disk or point ImageMagick to another disk with plenty of free space (with the MAGICK_TMPDIR environment variable).

[broucaries]

Thank you.

[broucaries]

The user checked if they are neouth space on the temporary dir and check quota, It seems ok but th job need more than 2Go of memory

We asked for a strace (http://bugs.debian.org/cgi-bin/bugrepor ... bug=349200)

Could you check if something goes wrong, and perhaps give us some debugging tips? Next step will be to ask for a ltrace, using debugging symbol I suppose ?

Regards

Bastien

[magick]

Try this command

• montage -limit memory 0 -limit map 0 -label %f -background '#fff' -geometry +4+4 -resize 10% '*_*.jpg' index.jpg

That forces the entire pixel cache to disk. See http://www.imagemagick.org/script/architecture.php for a discussion of the pixel cache and resource limits.

[pmarek]

Hello everybody,

I'm the user with the montage problem.

I just started

Code: Select all

MAGICK_TMPDIR=. montage -limit memory 0 -limit map 0 -label %f -background '#fff' -geometry +4+4 -resize 10% '*_*.jpg' index.jpg

as requested, in a directory with 68 jpgs (200MB).

After 10 minutes real time I get 66MB memory usage of montage, and the directory is at 4GB used.

Why is that so? I had estimated 68 files, with destination size about 350x250, gives for RGB (24bit) uncompressed about 18MB memory usage - which should very easily fit into main memory.

I now stopped montage, after about 2min user/16 min real time.
A short test shows that this should run much faster:

Code: Select all

$ time convert -label %f -background '#fff' -geometry +4+4 -resize 10%   20080921_142956.jpg aaa.jpg
real    0m1.897s
user    0m1.588s
sys     0m0.128s
$ identify aaa.jpg
aaa.jpg JPEG 326x245 326x245+0+0 DirectClass 8-bit 77.3867kb
flip@heim:/home/allgemein/pictures/2008/2008-09$ ls -la aaa.jpg
-rw-r--r-- 1 flip users 79244 13. Jän 20:46 aaa.jpg

Why does convert/montage of some files take so much more memory and time?

Thank you for your time and effort!

[magick]

Try this command (assumes thumbnails at 128 pixels):

• montage -label %f -background '#fff' -size 128x128 -geometry 128x128+4+4 '*_*.jpg' index.jpg

[pmarek]

Thank you, that seems to work nicely.

Is there some explanation that I can understand?

Thank you very much!


Regards,

Phil

[magick]

The size option causes ImageMagick to only store JPEG thumbnails in memory while its generating the montage layout. Without this option, the entire image is temporarily stored in memory until the montage page is generated.

[broucaries]

Could this usage documented ? It is really a user trap !

Regards

This command should be included in the exemple section of theman page with a note explaining why dropping -size could fail.

Bastien

[magick]

This feature is documented here: http://www.imagemagick.org/Usage/formats/#jpg_read. In addition we have someone (Rick Mabry) editting the documentation now. It may take several weeks because of our other responsibilities.

[broucaries]

thank you for your answer.

Regards

[rmabry]

In addition we have someone (Rick Mabry) editing the documentation now. It may take several weeks because of our other responsibilities.

It will take many, many weeks. Maybe years. Also, I am presently not editing anything beyond the command-line-options and command-line-processing pages.

Rick