enormous memory usage while resizing some images...

[Fluke571]

Hello,
today one of the webservers I administer started swapping heavily. After initial checking, I discovered imagemagick (6.3.1_1) caused it. "convert" process was using all available ram+swap.

USER PID %CPU %MEM VSZ RSS TTY STAT START TIME COMMAND
xxx 19267 16.0 52.3 2176260 538700 ? R 18:06 4:11 /usr/bin/convert /path/to/input.png -resize 100x /path/to/output.png

It seems to be easily reproducible, strace shows:
open("/usr/lib/ImageMagick-6.3.1/config/english.xml", O_RDONLY|O_LARGEFILE) = 4
_llseek(4, 0, [52413], SEEK_END) = 0
brk(0x80dd000) = 0x80dd000
mmap2(NULL, 52413, PROT_READ, MAP_PRIVATE, 4, 0) = 0xb7b50000
munmap(0xb7b50000, 52413) = 0
close(4) = 0
brk(0x80fe000) = 0x80fe000
brk(0x8122000) = 0x8122000
brk(0x814a000) = 0x814a000
brk(0x8172000) = 0x8172000
brk(0x819a000) = 0x819a000
brk(0x81c2000) = 0x81c2000
... etc

Two files that caused this behaviour are available at http://fluke.krneki.org/im/bug.zip

[magick]

We converted your images without any problems (no large memory requirements). We're using ImageMagick 6.3.1-6. In the mean-time, add -limit memory 64 -limit map 128 to restrict the amount of memory ImageMagick consumes.

[Fluke571]

update... I upgraded to latest IM (6.3.1_6) and now it seems to segfault:


strace convert /path/to/input.png -resize 100x /path/to/output.png
...
...
...
brk(0x82bc000) = 0x82bc000
brk(0x82dd000) = 0x82dd000
brk(0x82fe000) = 0x82fe000
brk(0x8320000) = 0x8320000
brk(0x8341000) = 0x8341000
--- SIGSEGV (Segmentation fault) @ 0 (0) ---
+++ killed by SIGSEGV +++

[Fluke571]

hm... limit option does not work for 6.3.1_1:

# convert -limit memory 64 -limit map 128 0.png -resize 100x y.png

and after 30s I get:

xxx 32520 98.5 7.5 317156 314708 pts/1 R+ 14:12 0:33 convert -limit memory 64 -limit map 128 in.png -resize 100x out.png

strace is constantly outputing these:
brk(0xb3b5000) = 0xb3b5000
brk(0xb3dd000) = 0xb3dd000
brk(0xb405000) = 0xb405000

[magick]

We cannot reproduce the problem. Try

• gdb convert
  run -limit memory 64 -limit map 128 0.png -resize 100x y.png
  where

and post the stack trace.

[Fluke571]

# gdb convert
GNU gdb 6.3
Copyright 2004 Free Software Foundation, Inc.
GDB is free software, covered by the GNU General Public License, and you are
welcome to change it and/or distribute copies of it under certain conditions.
Type "show copying" to see the conditions.
There is absolutely no warranty for GDB. Type "show warranty" for details.
This GDB was configured as "i486-slackware-linux"...(no debugging symbols found)
Using host libthread_db library "/lib/tls/libthread_db.so.1".

(gdb) run -limit memory 64 -limit map 128 0.png -resize 100x y.png
Starting program: /usr/bin/convert -limit memory 64 -limit map 128 0.png -resize 100x y.png
(no debugging symbols found)
(no debugging symbols found)
(no debugging symbols found)
(no debugging symbols found)
(no debugging symbols found)
[Thread debugging using libthread_db enabled]
[New Thread -1212586304 (LWP 1781)]

Program received signal SIGSEGV, Segmentation fault.
[Switching to Thread -1212586304 (LWP 1781)]
0xb7bc0882 in adler32 () from /usr/lib/libz.so.1
(gdb) where
#0 0xb7bc0882 in adler32 () from /usr/lib/libz.so.1
#1 0xb7bc592b in deflateCopy () from /usr/lib/libz.so.1
#2 0xb7bc569c in deflateCopy () from /usr/lib/libz.so.1
#3 0xb7bc389b in deflate () from /usr/lib/libz.so.1
#4 0xb7b6be11 in png_write_chunk_end () from /usr/lib/libpng12.so.0
#5 0xb7b6c897 in png_write_chunk_end () from /usr/lib/libpng12.so.0
#6 0xb7b70ace in png_write_info_before_PLTE () from /usr/lib/libpng12.so.0
#7 0xb7b70c07 in png_write_info () from /usr/lib/libpng12.so.0
#8 0xb7b8d882 in UnregisterPNGImage () from /usr/lib/ImageMagick-6.3.1/modules-Q16/coders/png.so
#9 0xb7b8e8e3 in UnregisterPNGImage () from /usr/lib/ImageMagick-6.3.1/modules-Q16/coders/png.so
#10 0xb7e886ce in WriteImage () from /usr/lib/libMagick.so.10
#11 0xb7e88a3f in WriteImages () from /usr/lib/libMagick.so.10
#12 0xb7dfcf8d in ConvertImageCommand () from /usr/lib/libWand.so.10
#13 0x0804898f in ?? ()
#14 0x0804a388 in ?? ()
#15 0x0000000b in ?? ()
#16 0x0804e4d8 in ?? ()
#17 0x00000000 in ?? ()
#18 0xbfc9dc4c in ?? ()
#19 0x00000000 in ?? ()
#20 0x00000000 in ?? ()
#21 0x00000000 in ?? ()
#22 0x00000000 in ?? ()
#23 0x00000000 in ?? ()
#24 0x0804a040 in ?? ()
#25 0x00000000 in ?? ()
#26 0x00000000 in ?? ()
#27 0xabacadab in ?? ()
#28 0xb7d7e000 in ?? () from /lib/tls/libc.so.6
#29 0xb7d8019c in nullserv () from /lib/tls/libc.so.6
#30 0x0000000b in ?? ()
#31 0xbfc9dcd8 in ?? ()
#32 0xb7c7afcb in __libc_start_main () from /lib/tls/libc.so.6
#33 0xb7c7afcb in __libc_start_main () from /lib/tls/libc.so.6
#34 0x080487d1 in ?? ()
(gdb)

[magick]

Something is corrupt with your ImageMagick build or your PNG delegate library. The stack trace shows the fault in the PNG library. Try uninstalling all versions of ImageMagick and/or libpng. When you reinstall, try building ImageMagick from source rather than any binary/RPM install. If it still fails, we're clueless.

[Fluke571]

these are all custom-builded (slackware) packages, and both versions are linked with libpng 1.2.14, along with glibc 2.3.6 and gcc 3.3.6

I tried downgrading libpng back to 1.2.12 and it works, however this version of libpng have a bug that might cause denial of service...

Now I see libpng 1.2.15 is already out, I'm compiling it right now and I'll post results in few minutes...

[Fluke571]

both 6.3.1_1 and 6.3.1_6 work ok with libpng 1.2.15...

---

I'm guessing this was it (from libpng ChangeLog):

version 1.2.15rc4 [December 27, 2006]
Fixed handling of rgb_to_gray when png_ptr->color.gray isn't set.

version 1.2.15rc5 [December 31, 2006]
Revised handling of rgb_to_gray.