PSD->PDF, Mac OS X 10.6.4, 64bit, very large memory usage

[darkart]

We're seeing an issue with ImageMagick-6.6.4-9 converting PSD files to PDF - at least one file is consuming a very large amount of memory (20+G of private mem). Trying to run 'identify +ping IMAGEFILE.psd' results in a crash (see below for crashed thread stack). We were able to convert this image with ImageMagick 6.6.3-0. We see the same issue w/ ImageMagick-6.6.4-8, and have not tested versions between 6.6.3-0 and 6.6.4-8 to further isolate.
Stack trace for 6.6.4-9 'identify +ping IMAGEFILE.psd' shows:

Thread 10 Crashed:
0 libMagickCore.4.dylib 0x0000000100023299 QueueAuthenticNexus + 553
1 libMagickCore.4.dylib 0x00000001000260af QueueCacheViewAuthenticPixels + 111
2 libMagickCore.4.dylib 0x00000001000d0142 SetImageBackgroundColor.omp_fn.3 + 130
3 libMagickCore.4.dylib 0x0000000100195b46 gomp_thread_start + 230
4 libSystem.B.dylib 0x00007fff80057456 _pthread_start + 331
5 libSystem.B.dylib 0x00007fff80057309 thread_start + 13


As we cannot provide this image for testing (sorry), is there anything else we can do to help diagnose and resolve this issue?

[magick]

If you don't need the layers you can always just extract the composite:

• convert 'image.psd[0]' image.png

If you need the layers, add -limit area 0 to your command line:

• convert -limit area 0 image.psd image-%02d.png

We cannot comment further without downloading the image and reproducing the problem.

[darkart]

Converting just the composite does work, however we need to have the layers as well. Adding the '-limit area 0' still results in the same crash. Trying to get other layers (convert image.psd[1] image.pdf) crashes as well - all crashes have the same stack trace, just different threads.
Unfortunately we cannot provide the image in question. If there are any tests we can run please let us know.

And, of course, thanks for your time and efforts!

[magick]

We're always interested in fixing problems-- if possible. Would it be possible to send the image privately to one of the ImageMagick developers rather than posting the image publicly? Would it be possible to use Photoshop to blank out the image and its layers so we cannot see the original image but it still causes ImageMagick to fail? If not, we'll close this issue.

[fmw42]

I don't know if this would be relevant or helpful to the IM developers, but you might use exiftool to get a dump of the PSD file and provide that.

exiftool -s -ee -g1 -u -n -D image.psd

[magick]

Unfortunately that would not be helpful. IM is misreading the layers and we need the image to determine why / where its going off track.

[darkart]

Bingo. The PSD mentioned in:

viewtopic.php?f=3&t=16721&p=61626&hilit=psd#p61626

also causes this crash. The image is available from:

https://bugs.launchpad.net/ubuntu/+sour ... bug/597469

or directly at:

https://bugs.launchpad.net/ubuntu/+sour ... k-hang.psd

I get the same stack trace and the same crash reason running 'identify +ping ubuntu-imagemagick-hang.psd' as with the image we have.

[magick]

Apparently some PSD layers have a negative height. We added a patch in ImageMagick 6.6.4-10 Beta available by sometime tomorrow to account for negative heights. Let us know if that fixes the problem with your PSD image.

[darkart]

I confirmed that 6.6.4-10 beta works to run 'identify +ping ubuntu-imagemagick-hang.psd'. I'll confirm against our data as soon as I can.

[darkart]

Unfortunately 6.6.4-10 beta (from a couple of days ago) still fails on our particular image. I have debug output that I can send to someone, it appears that identify is trying to allocate ~2.88 TB on disk which then fails and then identify crashes. Please let me know the best way to get the debug output to you, and/or if there's anything else we can do to help.

[magick]

The best thing to do is to make the image available for us to download so we can reproduce the problem. If the image is proprietary, use Photoshop to turn the layers into a solid or random pixels. You could try posting a URL to the debugging output but we suspect that will be insufficient for us to track down the source of the bug.

[darkart]

I've posted the debug log to:

http://www.darkart.com/ImageMagick_iden ... 8.1806.txt

The particular things I noticed were at:

lines 393 and 396 where there's an entry for "Area: 2.88622TB/2.88622TB/4.295GB" (that 4.295GB was present above too) and for "Disk: 2.88622TB/2.88622TB/18.446744EB" (yes, EB)

and then at line 404:

extend FILENAME.psd[0] (magick-TeTpzBvc[4], disk, 2.88622TB)

That's far too large of a space for this disk, and then it appears that identify crashes when it runs the disk out of space.

Sadly, I can't provide the image in question (I'd rather, it'd make this whole process about a bajillion times easier), even if I were to scrub the data in the image. I can, however, continue to examine the image as needed.

[magick]

Change

Code: Select all

            layer_info[i].page.y=(int) ReadBlobMSBLong(image);
            layer_info[i].page.x=(int) ReadBlobMSBLong(image);
            layer_info[i].page.height=(ssize_t)
              (ReadBlobMSBLong(image)-layer_info[i].page.y);
            layer_info[i].page.width=(ssize_t)
              (ReadBlobMSBLong(image)-layer_info[i].page.x);

to

Code: Select all

            int
              x,
              y;

            layer_info[i].page.y=(int) ReadBlobMSBLong(image);
            layer_info[i].page.x=(int) ReadBlobMSBLong(image);
            y=(int) ReadBlobMSBLong(image);
            x=(int) ReadBlobMSBLong(image);
            layer_info[i].page.height=(ssize_t) (y-layer_info[i].page.y);
            layer_info[i].page.width=(ssize_t) (x-layer_info[i].page.x);
            fprintf(stderr,"%d: %d,%d %d,%d %d,%d\n",i,  layer_info[i].page.x, layer_info[i].page.y, x, y,  layer_info[i].page.width,layer_info[i].page.height );

and post the output here.

[darkart]

The short answer is that that patch appears to resolve the issue, identify +ping FILENAME.psd worked.

Here's the output after patching psd.c:

identify +ping FILENAME.psd
0: 0,0 135,181 135,181
1: 0,0 0,0 0,0
2: 0,0 0,0 0,0
3: -10,-20 140,255 150,275
4: -10,-20 140,255 150,275
5: 0,0 0,0 0,0
6: -21,61 44,125 65,64
7: -65,75 -24,117 41,42
8: 73,79 102,107 29,28
9: 31,63 73,104 42,41
10: 140,61 182,102 42,41
11: 91,-206 127,-171 36,35
12: -167,-9 -132,26 35,35
13: 86,64 122,100 36,36
14: 0,0 0,0 0,0
15: 6,117 119,140 113,23
16: 6,118 116,140 110,22
17: 57,157 71,168 14,11
18: 6,146 112,168 106,22
19: 6,21 103,60 97,39
20: 0,0 0,0 0,0
21: -157,75 273,100 430,25
22: 74,38 187,123 113,85
23: -456,75 -201,101 255,26
24: -207,73 74,99 281,26
25: -207,77 71,95 278,18
26: 54,83 63,92 9,9
27: 5,82 53,93 48,11
28: 5,82 53,93 48,11
29: 5,111 130,112 125,1
30: 0,0 0,0 0,0
31: 0,0 0,0 0,0
32: 0,0 0,0 0,0
33: 0,8 135,57 135,49
34: 0,43 135,173 135,130
35: 0,0 0,0 0,0
36: 0,0 0,0 0,0
37: 0,111 135,172 135,61
38: 0,109 135,172 135,63
FILENAME.psd[0] PSD 135x181 135x181+0+0 8-bit DirectClass 737KB 0.080u 0:00.049
FILENAME.psd[1] PSD 135x181 135x181+0+0 8-bit DirectClass 737KB 0.070u 0:00.039
FILENAME.psd[2] PSD 150x275 150x275-10-20 8-bit DirectClass 737KB 0.070u 0:00.039
FILENAME.psd[3] PSD 150x275 150x275-10-20 8-bit DirectClass 737KB 0.070u 0:00.039
FILENAME.psd[4] PSD 65x64 65x64-21+61 8-bit DirectClass 737KB 0.060u 0:00.029
FILENAME.psd[5] PSD 41x42 41x42-65+75 8-bit DirectClass 737KB 0.050u 0:00.029
FILENAME.psd[6] PSD 29x28 29x28+73+79 8-bit DirectClass 737KB 0.050u 0:00.029
FILENAME.psd[7] PSD 42x41 42x41+31+63 8-bit DirectClass 737KB 0.050u 0:00.029
FILENAME.psd[8] PSD 42x41 42x41+140+61 8-bit DirectClass 737KB 0.050u 0:00.029
FILENAME.psd[9] PSD 36x35 36x35+91-206 8-bit DirectClass 737KB 0.050u 0:00.029
FILENAME.psd[10] PSD 35x35 35x35-167-9 8-bit DirectClass 737KB 0.050u 0:00.029
FILENAME.psd[11] PSD 36x36 36x36+86+64 8-bit DirectClass 737KB 0.050u 0:00.029
FILENAME.psd[12] PSD 113x23 113x23+6+117 8-bit DirectClass 737KB 0.040u 0:00.029
FILENAME.psd[13] PSD 110x22 110x22+6+118 8-bit DirectClass 737KB 0.040u 0:00.029
FILENAME.psd[14] PSD 14x11 14x11+57+157 8-bit DirectClass 737KB 0.040u 0:00.029
FILENAME.psd[15] PSD 106x22 106x22+6+146 8-bit DirectClass 737KB 0.040u 0:00.029
FILENAME.psd[16] PSD 97x39 97x39+6+21 8-bit DirectClass 737KB 0.040u 0:00.029
FILENAME.psd[17] PSD 430x25 430x25-157+75 8-bit DirectClass 737KB 0.040u 0:00.019
FILENAME.psd[18] PSD 113x85 113x85+74+38 8-bit DirectClass 737KB 0.040u 0:00.019
FILENAME.psd[19] PSD 255x26 255x26-456+75 8-bit DirectClass 737KB 0.040u 0:00.019
FILENAME.psd[20] PSD 281x26 281x26-207+73 8-bit DirectClass 737KB 0.040u 0:00.019
FILENAME.psd[21] PSD 278x18 278x18-207+77 8-bit DirectClass 737KB 0.030u 0:00.019
FILENAME.psd[22] PSD 9x9 9x9+54+83 8-bit DirectClass 737KB 0.030u 0:00.019
FILENAME.psd[23] PSD 48x11 48x11+5+82 8-bit DirectClass 737KB 0.030u 0:00.019
FILENAME.psd[24] PSD 48x11 48x11+5+82 8-bit DirectClass 737KB 0.030u 0:00.019
FILENAME.psd[25] PSD 125x1 125x1+5+111 8-bit DirectClass 737KB 0.030u 0:00.019
FILENAME.psd[26] PSD 135x49 135x49+0+8 8-bit DirectClass 737KB 0.020u 0:00.019
FILENAME.psd[27] PSD 135x130 135x130+0+43 8-bit DirectClass 737KB 0.010u 0:00.019
FILENAME.psd[28] PSD 135x61 135x61+0+111 8-bit DirectClass 737KB 0.010u 0:00.009
FILENAME.psd[29] PSD 135x63 135x63+0+109 8-bit DirectClass 737KB 0.010u 0:00.009

[darkart]

As well, we can now convert the image (psd) to PDF.