Deadlock when multiple threads write PDFs?

[caddr]

I have a program which is converting JPEGs to PDFs with Magick++. I have four threads running the same conversion code on different JPEGs. All for threads stop with the following backtrace:

Code: Select all

#0  0xb682e823 in ?? () from /usr/lib/libgomp.so.1
#1  0xb682e87f in ?? () from /usr/lib/libgomp.so.1
#2  0xb682d8ae in ?? () from /usr/lib/libgomp.so.1
#3  0xb6b7d5f6 in ?? () from /usr/lib/libMagickCore.so.1
#4  0xb6b7dab6 in ResizeImage () from /usr/lib/libMagickCore.so.1
#5  0xb6b7ddc5 in ZoomImage () from /usr/lib/libMagickCore.so.1
#6  0xb6b7e070 in ThumbnailImage () from /usr/lib/libMagickCore.so.1
#7  0xb576f165 in ?? () from /usr/lib/ImageMagick-6.4.5/modules-Q16/coders/pdf.so
#8  0xb6aa8863 in WriteImage () from /usr/lib/libMagickCore.so.1
#9  0xb7ecb112 in Magick::Image::write () from /usr/lib/libMagick++.so.1
#10 0x0807056a in image2pdf ...

where image2pdf essentially is Magick::Image(src).write(dst) (where src is a JPEG filename and dst is a PDF filename but with error handling). I'm testing on both Windows and Linux, and I've seen this issue on both platforms. I'm doing my threading with Qt if that's relevant. Any help resolving this issue would be appreciated.

[magick]

Download / install / run ImageMagick 6.5.4-10, the latest release. Does it deadlock as well? If so, let us know and we will investigate further.

[caddr]

Download / install / run ImageMagick 6.5.4-10, the latest release. Does it deadlock as well? If so, let us know and we will investigate further.

I just downloaded ImageMagick-6.5.5-0 source, compiled, and installed. It seems that the issue has been resolved (since 6.4.5.4 which was available in Ubuntu) as it did not deadlock. I will verify that it also works with the latest version in Windows soon and report back if it is still deadlocking.

[caddr]

I just downloaded ImageMagick-6.5.5-0 source, compiled, and installed. It seems that the issue has been resolved (since 6.4.5.4 which was available in Ubuntu) as it did not deadlock. I will verify that it also works with the latest version in Windows soon and report back if it is still deadlocking.

I may have spoke too soon. It seems as though they've locked up after all although it seemed like it was much less likely to happen as only two threads deadlocked while another two completed. Since I compiled from source, the backtrace is slightly more detailed (and is also somewhat different in this version from the previous):

Code: Select all

#0  0xb68a8823 in ?? () from /usr/lib/libgomp.so.1
#1  0xb68a887f in ?? () from /usr/lib/libgomp.so.1
#2  0xb68a78ae in ?? () from /usr/lib/libgomp.so.1
#3  0xb6c0a7c7 in SampleImage (image=0x806270e8, columns=410, rows=530, exception=0x8062a2b0) at magick/resize.c:1903
#4  0xb6c0d27c in ThumbnailImage (image=0x806270e8, columns=82, rows=106, exception=0x8062a2b0) at magick/resize.c:2908
#5  0xb49f7258 in WritePDFImage (image_info=0x746367b8, image=0x806270e8) at coders/pdf.c:1792
#6  0xb6b2c386 in WriteImage (image_info=0x91690338, image=0x806270e8) at magick/constitute.c:1125
#7  0xb7f62352 in Magick::Image::write (this=0xb36fe208, imageSpec_=@0xb36fe21c) at Magick++/lib/Image.cpp:2083
#8  0x080704ea in image2pdf

The above backtrace is from using ImageMagick-6.5.5-0, compiled from source in Ubuntu. Hopefully the more detailed info can help resolve the issue.

[magick]

Try adding --disable-openmp to your configure command line then rebuild and reinstall ImageMagick. Let's see if its a bug in libgomp.

[caddr]

Try adding --disable-openmp to your configure command line then rebuild and reinstall ImageMagick. Let's see if its a bug in libgomp.

Disabling OpenMP does seem to eliminate the deadlocks on my first attempt. I'm running my program again to be sure. Using ImageMagick with OpenMP disabled is a solution for me since I'm using multiple conversions to take advantage of multicore systems, but it would be nice to see this resolved even with it enabled.

[magick]

We disabled OpenMP for SampleImage() only in the current release, 6.5.5-1. We recommend you download it and leave OpenMP enabled. The deadlock problem should be resolved.

[caddr]

We disabled OpenMP for SampleImage() only in the current release, 6.5.5-1. We recommend you download it and leave OpenMP enabled. The deadlock problem should be resolved.

Unfortunately, it looks like there was nothing special about SampleImage that was causing a deadlock. The problem has moved and now I have a deadlock between two threads, but in two slightly different places:

Code: Select all

#0  0xb69cf823 in ?? () from /usr/lib/libgomp.so.1
#1  0xb69cf87f in ?? () from /usr/lib/libgomp.so.1
#2  0xb69ce8ae in ?? () from /usr/lib/libgomp.so.1
#3  0xb6d33076 in VerticalFilter (resize_filter=0xb4ad78f8, image=0xb4ad11e0, resize_image=0xb068acb0, y_factor=0.20000000000000001, span=188, quantum=0xb40a1c20, exception=0xb2ab2788)
    at magick/resize.c:1903
#4  0xb6d3425e in ResizeImage (image=0xb4ad11e0, columns=4294966784, rows=4294966784, filter=UndefinedFilter, blur=1, exception=0xb2ab2788) at magick/resize.c:2231
#5  0xb6d34555 in ZoomImage (image=0xb4ad11e0, columns=82, rows=106, exception=0xb2ab2788) at magick/resize.c:3012
#6  0xb6d34c0a in ThumbnailImage (image=0xb2aaf5c0, columns=82, rows=106, exception=0xb2ab2788) at magick/resize.c:2906
#7  0xb37b4258 in WritePDFImage (image_info=0xaccb3108, image=0xb2aaf5c0) at coders/pdf.c:1792
#8  0xb6c53476 in WriteImage (image_info=0xacaf9e28, image=0xb2aaf5c0) at magick/constitute.c:1125
#9  0xb8089352 in Magick::Image::write (this=0xb40ac208, imageSpec_=@0xb40ac21c) at Magick++/lib/Image.cpp:2083
#10 0x080704ea in image2pdf

and

Code: Select all

#0  0xb69cf823 in ?? () from /usr/lib/libgomp.so.1
#1  0xb69cf87f in ?? () from /usr/lib/libgomp.so.1
#2  0xb69ce8ae in ?? () from /usr/lib/libgomp.so.1
#3  0xb6d33eb6 in HorizontalFilter (resize_filter=0xc389208, image=0xb13e688, resize_image=0x9f74390, x_factor=0.20000000000000001, span=188, quantum=0xaebe9c20, exception=0xcf437e0) at magick/resize.c:1903
#4  0xb6d34293 in ResizeImage (image=0xe9b4ed8, columns=4294966784, rows=4294966784, filter=UndefinedFilter, blur=1, exception=0xcf437e0) at magick/resize.c:2233
#5  0xb6d34555 in ZoomImage (image=0xe9b4ed8, columns=82, rows=106, exception=0xcf437e0) at magick/resize.c:3012
#6  0xb6d34c0a in ThumbnailImage (image=0xcf40618, columns=82, rows=106, exception=0xcf437e0) at magick/resize.c:2906
#7  0xb37b4258 in WritePDFImage (image_info=0xa9433a0, image=0xcf40618) at coders/pdf.c:1792
#8  0xb6c53476 in WriteImage (image_info=0xa93b388, image=0xcf40618) at magick/constitute.c:1125
#9  0xb8089352 in Magick::Image::write (this=0xaebf4208, imageSpec_=@0xaebf421c) at Magick++/lib/Image.cpp:2083
#10 0x080704ea in image2pdf

So it seems that its a general issue with OpenMP rather than something specifically wrong with one code path. Maybe OpenMP isn't thread safe in the way that I'm using it?

[magick]

It could very well be your release version of OpenMP. We're using ImageMagick 6.5.5-2 with libgomp-4.4.1-6 with OpenMP enabled with 10 threads while sampling an image. We started the threads 10,000 times and the process completed without any deadlocks.

[caddr]

I'm running libgomp 4.3.3-5, but to be clear, I have 4 threads (separate from OpenMP threads) which are all converting separate JPEGs into PDFs concurrently. Would that be supported? From what I understand, in the test you mention, you're only doing one conversion at a time (but with several OpenMP threads).

[tisepti]

I have encountered this error as well;
libmagick6.4.5.4
libgomp4.3.3
(with the ubuntu 9.04 patches if applicable)


We are doing jpeg manipulation; not anything to do with pdfs. In our case it blocks for CropImage and SetImageBackgroundColor's invocation of GOMP_Parallell_start

[magick]

It does appear that a few versions of libgomp have bugs that can cause a deadlock within ImageMagick. In most cases libgomp works very well permitting ImageMagick algorithms to split work across all the cores of your host computer. However, in a few cases a dead lock occurs and our recommendation when this problem occurs is to with install a different version of libgomp or rebuild ImageMagick and disable OpenMP support:

• cd ImageMagick-6.3.6-4
  ./configure --disable-openmp
  make
  make install