[SOLVED] Magick++ API / OpenMP - special setup? Only one core used

whatdoido · Post by **whatdoido** » 2017-09-30T08:23:16-07:00

Summary: Magick++ API resize only uses one core/anything special required to enable OpenMP when using API.

EDIT: Solution
Add call to

Code: Select all

Magick::initializeMagick("");

before any other Magick calls - the documentation states this is NOT necessary for Unix/Linux systems but this appears not to be the case.

--- original post ---

I have a simple C++ util on linux that does some very basic Magick operations - essentially reads a (jpeg or tiff) file/creates Magick::Image and then resizes the image and writes to the results to a file as jpeg. I understand the resize operation can be an accelerated operation via OpenMP: https://www.imagemagick.org/script/openmp.php

The (unmod'd) code was recompiled/linked with gcc and with "-fopenmp -pthread".

However this isn't as fast as I expected and I see (via top) that only one core pushed to 98% whilst others idle. I have modified the policy file under /etc to explicit state the default thread numbers to 4/set MAGICK_THREAD_LIMIT=4 but no change (only one core used)

Running "convert -version", I see OpenMP is listed under features and running "convert logo: -resize 500% -bench 10 logo.png" I see all cores being used relatively evenly.

The is on Fedora 26 x86_64 with the version of ImageMagick being 6.9 direct from the official Fedora repos.

So I have two Qs:

is there a way to confirm the ImageMagick API/libraries are OpenMP enabled (not just the command line utils)

is there anything special required to enable OpenMP processing through the Magick++ API? additional header files, specific calls (ala Magick::enableOpenCL()), compiler directives etc.

The recompiled code has NO changes made except for the compile/link with additional flags to gcc.

Thanks

Post by **magick** » 2017-09-30T08:39:00-07:00

To determine if OpenMP is enabled, OpenMP is listed as a feature:

Code: Select all

$ convert -version
...
Features: Cipher DPC HDRI OpenMP

Magick++ is a wrapper around the MagickCore API's. If an image processing algorithm is threaded, it is supported from Magick++, the command-line, or any other ImageMagick language wrapper. Not all algorithms are threaded. None of the image readers or writers are threaded in ImageMagick although you can run many of them in separate threads in your code. Also, for the threaded algorithms, there must be enough work before an algorithm is threaded, e.g a 100x100 image won't thread because its more optimal to operate with just one thread.

whatdoido · Post by **whatdoido** » 2017-09-30T12:01:57-07:00

Thanks.

Yes, "convert" shows OpenMP as a listed feature.

The images I'm resizing (scaling down) are 7360x4912; is there anything else I can check to understand why only one CPU core is being stressed during the Magick::Image::resize() operation?

Post by **magick** » 2017-09-30T12:08:52-07:00

Magick::Image::resize() calls the MagickCore::ResizeImage() method which is threaded. Its the same method called by the -resize command-line option. Use GDB to see if any threads are enabled. Our gdb claimed 8 threads were created for 'convert logo: -resize 200% null:'

Post by **dlemstra** » 2017-09-30T13:33:06-07:00

It is possible that the GPU is being used. Did you compile IM with OpenCL support?

whatdoido · Post by **whatdoido** » 2017-10-01T00:49:16-07:00

I've put some timing code around the call to Magick::resize() and stuck this through gdb.

The call to resize() takes just over 2 seconds to return

Code: Select all

1154		magick.resize(Magick::Geometry(tmp));
Missing separate debuginfos, use: dnf debuginfo-install ImageMagick-c++-6.9.9.13-1.fc26.x86_64 ImageMagick-libs-6.9.9.13-1.fc26.x86_64 bzip2-libs-1.0.6-22.fc26.x86_64 exiv2-libs-0.25-4.fc26.x86_64 expat-2.2.4-1.fc26.x86_64 fftw-libs-double-3.3.5-4.fc26.x86_64 fontconfig-2.12.1-4.fc26.x86_64 freetype-2.7.1-9.fc26.x86_64 lcms2-2.8-3.fc26.x86_64 libICE-1.0.9-9.fc26.x86_64 libSM-1.2.2-5.fc26.x86_64 libX11-1.6.5-2.fc26.x86_64 libXau-1.0.8-7.fc26.x86_64 libXext-1.3.3-5.fc26.x86_64 libXt-1.1.5-4.fc26.x86_64 libgcc-7.2.1-2.fc26.x86_64 libgomp-7.2.1-2.fc26.x86_64 libjpeg-turbo-1.5.1-0.fc26.x86_64 libpng-1.6.28-2.fc26.x86_64 libstdc++-7.2.1-2.fc26.x86_64 libtool-ltdl-2.4.6-17.fc26.x86_64 libuuid-2.30.2-1.fc26.x86_64 libxcb-1.12-3.fc26.x86_64 xz-libs-5.2.3-2.fc26.x86_64 zlib-1.2.11-2.fc26.x86_64
(gdb) info threads
  Id   Target Id         Frame 
* 1    Thread 0x7ffff7fc1800 (LWP 4014) "leak" Exiv2GdkPxBufLdr::ImgFactory::create (this=0x63e3e0, buf_=0x7ffff1c10010 "II*", 
    bufsz_=22810286, imgbuf_=..., mimeType_="image/jpeg") at ImgFactory.cc:1154
(gdb) n
1155	            const std::chrono::duration<double>  elapsed = std::chrono::system_clock::now() - start;
(gdb) info threads
  Id   Target Id         Frame 
* 1    Thread 0x7ffff7fc1800 (LWP 4014) "leak" Exiv2GdkPxBufLdr::ImgFactory::create (this=0x63e3e0, buf_=0x7ffff1c10010 "II*", 
    bufsz_=22810286, imgbuf_=..., mimeType_="image/jpeg") at ImgFactory.cc:1155

I don't see any threads being created.

In /etc/ImageMagick-6/policy.xml I see

Code: Select all

<policymap>
  <policy domain="resource" name="memory" value="4GiB"/>
  <policy domain="resource" name="map" value="1GiB"/>
  <policy domain="resource" name="thread" value="4"/>
 </policymap>

The version of ImageMagick is from the Fedora 26 repos

Code: Select all

$ convert --version
Version: ImageMagick 6.9.9-13 Q16 x86_64 2017-09-14 http://www.imagemagick.org
Copyright: © 1999-2017 ImageMagick Studio LLC
License: http://www.imagemagick.org/script/license.php
Features: Cipher DPC Modules OpenMP
Delegates (built-in): bzlib cairo djvu fftw fontconfig freetype gslib jbig jng jp2 jpeg lcms ltdl lzma openexr pangocairo png ps rsvg tiff webp wmf x xml zlib

Post by **magick** » 2017-10-01T05:05:34-07:00

Try these commands:

Code: Select all

$ gdb convert
run logo: -resize 200% null:

Does gdb report any threads created? If not, we need to check the obvious. Does `cat /proc/cpuinfo` report more than one core

?

whatdoido · Post by **whatdoido** » 2017-10-01T08:54:38-07:00

Yes, convert spawns trheads

Code: Select all

(gdb) run logo: -resize 2005 null:
Starting program: /usr/bin/convert logo: -resize 2005 null:
[Thread debugging using libthread_db enabled]
Using host libthread_db library "/lib64/libthread_db.so.1".
[New Thread 0x7ffff0811700 (LWP 4684)]
[New Thread 0x7ffff189b700 (LWP 4685)]
[New Thread 0x7fffebfff700 (LWP 4686)]
[Thread 0x7fffebfff700 (LWP 4686) exited]
[Thread 0x7ffff189b700 (LWP 4685) exited]
[New Thread 0x7fffebfff700 (LWP 4687)]
[New Thread 0x7ffff189b700 (LWP 4688)]
[Thread 0x7ffff189b700 (LWP 4688) exited]
[Thread 0x7fffebfff700 (LWP 4687) exited]
[Thread 0x7ffff7fc37c0 (LWP 4683) exited]
[Inferior 1 (process 4683) exited normally]
(gdb)

The CPU is a i7 870 - 4x hyperthread cores

libgomp is also installed

Code: Select all

sudo dnf info libgomp
Last metadata expiration check: 0:49:49 ago on Sun 01 Oct 2017 16:09:45 BST.
Installed Packages
Name         : libgomp
Version      : 7.2.1
Release      : 2.fc26
Arch         : x86_64
Size         : 278 k
Source       : gcc-7.2.1-2.fc26.src.rpm
Repo         : @System
From repo    : updates
Summary      : GCC OpenMP v4.5 shared support library
URL          : http://gcc.gnu.org
License      : GPLv3+ and GPLv3+ with exceptions and GPLv2+ with exceptions and LGPLv2+ and BSD
Description  : This package contains GCC shared support library which is needed
             : for OpenMP v4.5 support.

and ldd against my binary shows libgomp as a dep

Post by **magick** » 2017-10-01T09:07:09-07:00

Ok, now try gdb with your Magick++ application. Does it spawn threads? If not, check the image dimensions. If its a small image, the resize will occur in 1 thread. One thread will also be used if the MAGICK_THREADS environment variable or thread security policy limits the number of threads. Another option is that your Magick++ library is linking to a non-threaded version of ImageMagick that exists on your system. And the final option is that you have entered an alternative reality where Magick++ does not thread but the command-line does, tread carefully.

whatdoido · Post by **whatdoido** » 2017-10-01T09:45:58-07:00

Running convert and checking its loaded shared libs:

Code: Select all

(gdb) run logo: -resize 200% null:
Starting program: /usr/bin/convert logo: -resize 200% null:
[Thread debugging using libthread_db enabled]
Using host libthread_db library "/lib64/libthread_db.so.1".
[New Thread 0x7fffec5f3700 (LWP 12821)]
[New Thread 0x7fffebdf2700 (LWP 12822)]
[New Thread 0x7fffeb5f1700 (LWP 12823)]
[Thread 0x7fffeb5f1700 (LWP 12823) exited]
[Thread 0x7fffebdf2700 (LWP 12822) exited]
[Thread 0x7fffec5f3700 (LWP 12821) exited]
[Inferior 1 (process 12817) exited normally]
(gdb) sharedlibrary Magick
Symbols already loaded for /lib64/libMagickCore-6.Q16.so.2
Symbols already loaded for /lib64/libMagickWand-6.Q16.so.2

Running my process against a 5472x3648 image, scaling down to 4288 - it's acutally a pulling the embedded jpeg from a Canon RAW file and doing the Image::resize() on that.

Code: Select all

(gdb) set args /export/public/imgs/RAW_CANON_6D.CR2
(gdb) run
Starting program: /home/ray/dev/eog-plugin-exif-rating-raw-viewer/gdk-pixbuf/leak /export/public/imgs/RAW_CANON_6D.CR2
Missing separate debuginfos, use: dnf debuginfo-install glibc-2.25-10.fc26.x86_64
[Thread debugging using libthread_db enabled]
Using host libthread_db library "/lib64/libthread_db.so.1".
EXIV2_PIXBUF_LOADER_SCALE_LIMIT=4288
EXIV2_PIXBUF_LOADER_CONVERT_SRGB=1
EXIV2_PIXBUF_LOADER_ROTATE=1
[Inferior 1 (process 12537) exited normally]
Missing separate debuginfos, use: dnf debuginfo-install ImageMagick-c++-6.9.3.0-6.fc26.x86_64 ImageMagick-libs-6.9.3.0-6.fc26.x86_64 bzip2-libs-1.0.6-22.fc26.x86_64 exiv2-libs-0.25-4.fc26.x86_64 expat-2.2.4-1.fc26.x86_64 fftw-libs-double-3.3.5-4.fc26.x86_64 fontconfig-2.12.1-4.fc26.x86_64 freetype-2.7.1-9.fc26.x86_64 lcms2-2.8-3.fc26.x86_64 libICE-1.0.9-9.fc26.x86_64 libSM-1.2.2-5.fc26.x86_64 libX11-1.6.5-2.fc26.x86_64 libXau-1.0.8-7.fc26.x86_64 libXext-1.3.3-5.fc26.x86_64 libXt-1.1.5-4.fc26.x86_64 libgcc-7.1.1-3.fc26.x86_64 libgomp-7.1.1-3.fc26.x86_64 libjpeg-turbo-1.5.1-0.fc26.x86_64 libpng-1.6.28-2.fc26.x86_64 libstdc++-7.1.1-3.fc26.x86_64 libtool-ltdl-2.4.6-17.fc26.x86_64 libuuid-2.30.1-1.fc26.x86_64 libxcb-1.12-3.fc26.x86_64 xz-libs-5.2.3-2.fc26.x86_64 zlib-1.2.11-2.fc26.x86_64
9gdb) sharedlibrary Magick
Symbols already loaded for /lib64/libMagick++-6.Q16.so.6
Symbols already loaded for /lib64/libMagickWand-6.Q16.so.2
Symbols already loaded for /lib64/libMagickCore-6.Q16.so.2
Symbols already loaded for /usr/lib64/ImageMagick-6.9.3/modules-Q16/coders/jpeg.so
Symbols already loaded for /usr/lib64/ImageMagick-6.9.3/modules-Q16/coders/xc.so
(gdb)

No threads spawned, same magic core and wand loaded.

Nothing showing from:

Code: Select all

 env | grep -i MAGICK

Only reference to thread is in

Code: Select all

/etc/ImageMagick-6/policy.xml
<policy domain="resource" name="thread" value="4"/>

Anything else specific I should check?

Post by **snibgo** » 2017-10-01T09:54:32-07:00

It looks like convert and the user's program are using different libraries:

whatdoido wrote:Version: ImageMagick 6.9.9-13 Q16 x86_64

whatdoido wrote:ImageMagick-c++-6.9.3.0-6

whatdoido · Post by **whatdoido** » 2017-10-01T10:25:28-07:00

Some missing coffee needed - removed ImageMagick and ImageMagick-c++ and reinstalled and same tests again - same results. No threads in user program.

Code: Select all

$ convert --version
Version: ImageMagick 6.9.9-13 Q16 x86_64 2017-09-14 http://www.imagemagick.org
Copyright: © 1999-2017 ImageMagick Studio LLC
License: http://www.imagemagick.org/script/license.php
Features: Cipher DPC Modules OpenMP 
Delegates (built-in): bzlib cairo djvu fftw fontconfig freetype gslib jbig jng jp2 jpeg lcms ltdl lzma openexr pangocairo png ps rsvg tiff webp wmf x xml zlib
$ ldd /usr/bin/convert
        linux-vdso.so.1 (0x00007ffccad52000)
        libMagickCore-6.Q16.so.5 => /lib64/libMagickCore-6.Q16.so.5 (0x00007f1de89f3000)
        libMagickWand-6.Q16.so.5 => /lib64/libMagickWand-6.Q16.so.5 (0x00007f1de86d6000)
        liblcms2.so.2 => /lib64/liblcms2.so.2 (0x00007f1de847b000)
        libfftw3.so.3 => /lib64/libfftw3.so.3 (0x00007f1de807e000)
        libfontconfig.so.1 => /lib64/libfontconfig.so.1 (0x00007f1de7e3a000)
        libfreetype.so.6 => /lib64/libfreetype.so.6 (0x00007f1de7b88000)
        libXext.so.6 => /lib64/libXext.so.6 (0x00007f1de7976000)
        libXt.so.6 => /lib64/libXt.so.6 (0x00007f1de770e000)
        liblzma.so.5 => /lib64/liblzma.so.5 (0x00007f1de74e8000)
        libbz2.so.1 => /lib64/libbz2.so.1 (0x00007f1de72d8000)
        libz.so.1 => /lib64/libz.so.1 (0x00007f1de70c1000)
        libltdl.so.7 => /lib64/libltdl.so.7 (0x00007f1de6eb5000)
        libSM.so.6 => /lib64/libSM.so.6 (0x00007f1de6cad000)
        libICE.so.6 => /lib64/libICE.so.6 (0x00007f1de6a91000)
        libX11.so.6 => /lib64/libX11.so.6 (0x00007f1de6753000)
        libgomp.so.1 => /lib64/libgomp.so.1 (0x00007f1de6524000)
        libm.so.6 => /lib64/libm.so.6 (0x00007f1de620e000)
        libpthread.so.0 => /lib64/libpthread.so.0 (0x00007f1de5fed000)
        libc.so.6 => /lib64/libc.so.6 (0x00007f1de5c1c000)
        libgcc_s.so.1 => /lib64/libgcc_s.so.1 (0x00007f1de5a05000)
        libexpat.so.1 => /lib64/libexpat.so.1 (0x00007f1de57d2000)
        libpng16.so.16 => /lib64/libpng16.so.16 (0x00007f1de559f000)
        libdl.so.2 => /lib64/libdl.so.2 (0x00007f1de5399000)
        libuuid.so.1 => /lib64/libuuid.so.1 (0x00007f1de5194000)
        libxcb.so.1 => /lib64/libxcb.so.1 (0x00007f1de4f6c000)
        /lib64/ld-linux-x86-64.so.2 (0x0000564d59d0e000)
        libXau.so.6 => /lib64/libXau.so.6 (0x00007f1de4d68000)

user program deps:

Code: Select all

$ ldd leak
        linux-vdso.so.1 (0x00007fff95523000)
        libexiv2.so.14 => /lib64/libexiv2.so.14 (0x00007f8970f57000)
        libMagick++-6.Q16.so.8 => /lib64/libMagick++-6.Q16.so.8 (0x00007f8970ccf000)
        libMagickWand-6.Q16.so.5 => /lib64/libMagickWand-6.Q16.so.5 (0x00007f89709b2000)
        libMagickCore-6.Q16.so.5 => /lib64/libMagickCore-6.Q16.so.5 (0x00007f89704ec000)
        libstdc++.so.6 => /lib64/libstdc++.so.6 (0x00007f8970164000)
        libm.so.6 => /lib64/libm.so.6 (0x00007f896fe4c000)
        libgomp.so.1 => /lib64/libgomp.so.1 (0x00007f896fc1d000)
        libgcc_s.so.1 => /lib64/libgcc_s.so.1 (0x00007f896fa06000)
        libpthread.so.0 => /lib64/libpthread.so.0 (0x00007f896f7e7000)
        libc.so.6 => /lib64/libc.so.6 (0x00007f896f416000)
        /lib64/ld-linux-x86-64.so.2 (0x0000561fd6219000)
        libz.so.1 => /lib64/libz.so.1 (0x00007f896f1ff000)
        libexpat.so.1 => /lib64/libexpat.so.1 (0x00007f896efca000)
        libdl.so.2 => /lib64/libdl.so.2 (0x00007f896edc6000)
        liblcms2.so.2 => /lib64/liblcms2.so.2 (0x00007f896eb6b000)
        libfftw3.so.3 => /lib64/libfftw3.so.3 (0x00007f896e76e000)
        libfontconfig.so.1 => /lib64/libfontconfig.so.1 (0x00007f896e52a000)
        libfreetype.so.6 => /lib64/libfreetype.so.6 (0x00007f896e27a000)
        libXext.so.6 => /lib64/libXext.so.6 (0x00007f896e066000)
        libXt.so.6 => /lib64/libXt.so.6 (0x00007f896ddfe000)
        liblzma.so.5 => /lib64/liblzma.so.5 (0x00007f896dbd8000)
        libbz2.so.1 => /lib64/libbz2.so.1 (0x00007f896d9c8000)
        libltdl.so.7 => /lib64/libltdl.so.7 (0x00007f896d7be000)
        libSM.so.6 => /lib64/libSM.so.6 (0x00007f896d5b6000)
        libICE.so.6 => /lib64/libICE.so.6 (0x00007f896d398000)
        libX11.so.6 => /lib64/libX11.so.6 (0x00007f896d05a000)
        libpng16.so.16 => /lib64/libpng16.so.16 (0x00007f896ce27000)
        libuuid.so.1 => /lib64/libuuid.so.1 (0x00007f896cc22000)
        libxcb.so.1 => /lib64/libxcb.so.1 (0x00007f896c9f8000)
        libXau.so.6 => /lib64/libXau.so.6 (0x00007f896c7f4000)

Running convert through GDB - threads spawned

Code: Select all

$ gdb convert
GNU gdb (GDB) Fedora 8.0-24.fc26
Copyright (C) 2017 Free Software Foundation, Inc.
License GPLv3+: GNU GPL version 3 or later <http://gnu.org/licenses/gpl.html>
This is free software: you are free to change and redistribute it.
There is NO WARRANTY, to the extent permitted by law.  Type "show copying"
and "show warranty" for details.
This GDB was configured as "x86_64-redhat-linux-gnu".
Type "show configuration" for configuration details.
For bug reporting instructions, please see:
<http://www.gnu.org/software/gdb/bugs/>.
Find the GDB manual and other documentation resources online at:
<http://www.gnu.org/software/gdb/documentation/>.
For help, type "help".
Type "apropos word" to search for commands related to "word"...
Reading symbols from convert...Reading symbols from /home/ray/dev/eog-plugin-exif-rating-raw-viewer/gdk-pixbuf/convert...(no debugging symbols found)...done.
(no debugging symbols found)...done.
Missing separate debuginfos, use: dnf debuginfo-install ImageMagick-6.9.9.13-1.fc26.x86_64
(gdb) run logo: -resize 200% null:
Starting program: /usr/bin/convert logo: -resize 200% null:
[Thread debugging using libthread_db enabled]
Using host libthread_db library "/lib64/libthread_db.so.1".
[New Thread 0x7fffeb451700 (LWP 8739)]
[New Thread 0x7fffeac50700 (LWP 8744)]
[New Thread 0x7fffea44f700 (LWP 8746)]
[Thread 0x7fffea44f700 (LWP 8746) exited]
[Thread 0x7fffeac50700 (LWP 8744) exited]
[New Thread 0x7fffea44f700 (LWP 8755)]
[New Thread 0x7fffeac50700 (LWP 8756)]
[Thread 0x7fffeac50700 (LWP 8756) exited]
[Thread 0x7fffea44f700 (LWP 8755) exited]
[Thread 0x7ffff7fc57c0 (LWP 8591) exited]
[Inferior 1 (process 8591) exited normally]
(gdb) sharedlibrary Magick
Symbols already loaded for /lib64/libMagickCore-6.Q16.so.5
Symbols already loaded for /lib64/libMagickWand-6.Q16.so.5
(gdb)

Running user program - no threads

Code: Select all

$ gdb ./leak
GNU gdb (GDB) Fedora 8.0-24.fc26
Copyright (C) 2017 Free Software Foundation, Inc.
License GPLv3+: GNU GPL version 3 or later <http://gnu.org/licenses/gpl.html>
This is free software: you are free to change and redistribute it.
There is NO WARRANTY, to the extent permitted by law.  Type "show copying"
and "show warranty" for details.
This GDB was configured as "x86_64-redhat-linux-gnu".
Type "show configuration" for configuration details.
For bug reporting instructions, please see:
<http://www.gnu.org/software/gdb/bugs/>.
Find the GDB manual and other documentation resources online at:
<http://www.gnu.org/software/gdb/documentation/>.
For help, type "help".
Type "apropos word" to search for commands related to "word"...
Reading symbols from ./leak...(no debugging symbols found)...done.
(gdb) set args /export/public/imgs/RAW_CANON_6D.CR2
(gdb) run
Starting program: /home/ray/dev/eog-plugin-exif-rating-raw-viewer/gdk-pixbuf/leak /export/public/imgs/RAW_CANON_6D.CR2
Missing separate debuginfos, use: dnf debuginfo-install glibc-2.25-10.fc26.x86_64
[Thread debugging using libthread_db enabled]
Using host libthread_db library "/lib64/libthread_db.so.1".
EXIV2_PIXBUF_LOADER_SCALE_LIMIT=4288
EXIV2_PIXBUF_LOADER_CONVERT_SRGB=1
EXIV2_PIXBUF_LOADER_ROTATE=1
[Inferior 1 (process 17198) exited normally]
Missing separate debuginfos, use: dnf debuginfo-install ImageMagick-c++-6.9.9.13-1.fc26.x86_64 ImageMagick-libs-6.9.9.13-1.fc26.x86_64 bzip2-libs-1.0.6-22.fc26.x86_64 exiv2-libs-0.25-4.fc26.x86_64 expat-2.2.4-1.fc26.x86_64 fftw-libs-double-3.3.5-4.fc26.x86_64 fontconfig-2.12.1-4.fc26.x86_64 freetype-2.7.1-9.fc26.x86_64 lcms2-2.8-3.fc26.x86_64 libICE-1.0.9-9.fc26.x86_64 libSM-1.2.2-5.fc26.x86_64 libX11-1.6.5-2.fc26.x86_64 libXau-1.0.8-7.fc26.x86_64 libXext-1.3.3-5.fc26.x86_64 libXt-1.1.5-4.fc26.x86_64 libgcc-7.1.1-3.fc26.x86_64 libgomp-7.1.1-3.fc26.x86_64 libjpeg-turbo-1.5.1-0.fc26.x86_64 libpng-1.6.28-2.fc26.x86_64 libstdc++-7.1.1-3.fc26.x86_64 libtool-ltdl-2.4.6-17.fc26.x86_64 libuuid-2.30.1-1.fc26.x86_64 libxcb-1.12-3.fc26.x86_64 xz-libs-5.2.3-2.fc26.x86_64 zlib-1.2.11-2.fc26.x86_64
(gdb) sharedlibrary Magick
Symbols already loaded for /lib64/libMagick++-6.Q16.so.8
Symbols already loaded for /lib64/libMagickWand-6.Q16.so.5
Symbols already loaded for /lib64/libMagickCore-6.Q16.so.5
Symbols already loaded for /usr/lib64/ImageMagick-6.9.9/modules-Q16/coders/jpeg.so
Symbols already loaded for /usr/lib64/ImageMagick-6.9.9/modules-Q16/coders/xc.so
(gdb) quit

Same MagickWand and MagickCore libs

For those interested, user code:
https://github.com/whatdoineed2do/eog-p ... gdk-pixbuf

The file ImageFactory.cc:1156 is where the resize() is called from the stack leak.cc:379:main()

the Github doesn't have the -fopenmp flags. Building locally we see:

Code: Select all

$ make leak
g++ -c -fopenmp -pthread -fpermissive -Wconversion-null -Wno-write-strings  -fPIC  -I/usr/include/gdk-pixbuf-2.0 -I/usr/include/glib-2.0 -I/usr/lib64/glib-2.0/include -pthread -I/usr/include/libpng16   -I/usr/include/ImageMagick-6 -fopenmp -DMAGICKCORE_HDRI_ENABLE=0 -DMAGICKCORE_QUANTUM_DEPTH=16  leak.cc
g++ -c -fopenmp -pthread -fpermissive -Wconversion-null -Wno-write-strings  -fPIC  -I/usr/include/gdk-pixbuf-2.0 -I/usr/include/glib-2.0 -I/usr/lib64/glib-2.0/include -pthread -I/usr/include/libpng16   -I/usr/include/ImageMagick-6 -fopenmp -DMAGICKCORE_HDRI_ENABLE=0 -DMAGICKCORE_QUANTUM_DEPTH=16  ImgFactory.cc
g++ -c -fopenmp -pthread -fpermissive -Wconversion-null -Wno-write-strings  -fPIC  -I/usr/include/gdk-pixbuf-2.0 -I/usr/include/glib-2.0 -I/usr/lib64/glib-2.0/include -pthread -I/usr/include/libpng16   -I/usr/include/ImageMagick-6 -fopenmp -DMAGICKCORE_HDRI_ENABLE=0 -DMAGICKCORE_QUANTUM_DEPTH=16  DbgHlpr.cc
g++ -ggdb  -fopenmp -pthread  -I/usr/include/ImageMagick-6 -fopenmp -DMAGICKCORE_HDRI_ENABLE=0 -DMAGICKCORE_QUANTUM_DEPTH=16  leak.o ImgFactory.o DbgHlpr.o -lexiv2  -lMagick++-6.Q16 -lMagickWand-6.Q16 -lMagickCore-6.Q16  -lpthread -o leak

Is there somethign I'm missing on the compiler flags?

whatdoido · Post by **whatdoido** » 2017-10-01T10:53:08-07:00

using convert to resize the input jpeg spawns threads so we know the input should also have the user program do the same

Code: Select all

(gdb) run -resize 4288 RAW_CANON_6D.CR2.jpg foo.jpg
Starting program: /usr/bin/convert -resize 4288 RAW_CANON_6D.CR2.jpg foo.jpg
[Thread debugging using libthread_db enabled]
Using host libthread_db library "/lib64/libthread_db.so.1".
[New Thread 0x7fffd601d700 (LWP 29438)]
[New Thread 0x7fffd581c700 (LWP 29439)]
[New Thread 0x7fffd501b700 (LWP 29440)]
[Thread 0x7fffd501b700 (LWP 29440) exited]
[Thread 0x7fffd581c700 (LWP 29439) exited]
[Thread 0x7fffd601d700 (LWP 29438) exited]
[Inferior 1 (process 29437) exited normally]
(gdb)

whatdoido · Post by **whatdoido** » 2017-10-01T11:14:35-07:00

and the compiler is fine at generating openmp enabled code

Using code from here: https://people.sc.fsu.edu/~jburkardt/cp ... penmp.html

Code: Select all

 g++ -fopenmp -pthread ./prime_openmp.cpp -o openmp -lpthread

Code: Select all

(gdb) set environment OMP_NUM_THREADS=4
(gdb) run
Starting program: /home/ray/dev/eog-plugin-exif-rating-raw-viewer/gdk-pixbuf/openmp 
[Thread debugging using libthread_db enabled]
Using host libthread_db library "/lib64/libthread_db.so.1".

PRIME_OPENMP
  C++/OpenMP version

  Number of processors available = 4
  Number of threads =              4

TEST01
  Call PRIME_NUMBER to count the primes from 1 to N.

         N        Pi          Time

[New Thread 0x7ffff6afc700 (LWP 30092)]
[New Thread 0x7ffff62fb700 (LWP 30093)]
[New Thread 0x7ffff5afa700 (LWP 30094)]
         1         0      0.00483796
         2         1      1.1873e-05
         4         2       8.381e-06
         8         4       8.381e-06

Setting the OMP_NUM_THREAD on my program has no impact - still one thread

whatdoido · Post by **whatdoido** » 2017-10-01T12:46:52-07:00

This looks like a bug now:

Simplified sample program that does a resize - compile time flag to determine magick core calls or magick++ calls

Code: Select all

#include <sys/types.h>
#include <string.h>
#include <strings.h>

#include <iostream>

#include <omp.h>

using namespace  std;



#ifdef USE_MAGICK_PLUSPLUS
#include <Magick++.h>
void  _magick(const char* file_)
{
    cout << "using Magick++" << endl;

    Magick::Image  magick(file_ == NULL ? "6000x4000" : file_, "blue");
    magick.filterType(Magick::LanczosFilter);
    magick.quality(70);
    magick.resize(Magick::Geometry("x4288"));
}
#else
#include <magick/MagickCore.h>
void  _magick(const char* file_)
{
    cout << "using MagickCore" << endl;

    // from https://github.com/ImageMagick/ImageMagick/blob/d06c8899b44a0592ff7b6f4ad14a595b8de2084a/www/source/core.c
    ExceptionInfo
	*exception;

    Image
	*image,
	*images,
	*resize_image;

    ImageInfo
	*image_info;

    /*
       Initialize the image info structure and read an image.
       */
    MagickCoreGenesis(NULL, MagickTrue);
    exception=AcquireExceptionInfo();
    image_info=CloneImageInfo((ImageInfo *) NULL);
    strcpy(image_info->filename, file_);
    images = ReadImage(image_info,exception);

    if (exception->severity != UndefinedException)
	CatchException(exception);

    if (images == (Image *) NULL) {
	cout << "no image" << endl;
	return;
    }

    while ((image=RemoveFirstImageFromList(&images)) != (Image *) NULL)
    {
	resize_image = ResizeImage(image,4288,80,LanczosFilter,1.0,exception);
	if (resize_image == (Image *) NULL)
	    MagickError(exception->severity,exception->reason,exception->description);
	DestroyImage(image);
    }
}
#endif


int main(int argc, char* const argv[])
{
    int  ret = 0;
    try
    {
	 cout << "OMP: cpus=" << omp_get_num_procs() << " threads=" << omp_get_max_threads() << endl;
	_magick(argc == 2 ? argv[1] : NULL);
    }
    catch (const std::exception& ex)
    {
	cerr << "failed to magick - " << ex.what() << endl;
	ret = 1;
    }

    return ret;
}

compiling with:

Code: Select all

g++ -fopenmp -pthread -g  -I/usr/include/ImageMagick-6 -fopenmp -DMAGICKCORE_HDRI_ENABLE=0 -DMAGICKCORE_QUANTUM_DEPTH=16  -I/usr/include/ImageMagick-6 -fopenmp -DMAGICKCORE_HDRI_ENABLE=0 -DMAGICKCORE_QUANTUM_DEPTH=16  tests/mag.cc  -lMagickCore-6.Q16  -lMagick++-6.Q16 -lMagickWand-6.Q16 -lMagickCore-6.Q16  -lpthread -o mag-core

Code: Select all

g++ -DUSE_MAGICK_PLUSPLUS -fopenmp -pthread -g  -I/usr/include/ImageMagick-6 -fopenmp -DMAGICKCORE_HDRI_ENABLE=0 -DMAGICKCORE_QUANTUM_DEPTH=16  -I/usr/include/ImageMagick-6 -fopenmp -DMAGICKCORE_HDRI_ENABLE=0 -DMAGICKCORE_QUANTUM_DEPTH=16  tests/mag.cc  -lMagickCore-6.Q16  -lMagick++-6.Q16 -lMagickWand-6.Q16 -lMagickCore-6.Q16  -lpthread -o mag-plus

Running the debugger against the MagickCore, threads created.

Running the debugger against the Magick++ calls, no threads.

Same input used.

Is this Magick problem or a Fedora build problem? Can someone with a local build compile confirm?

Legacy ImageMagick Discussions Archive

[SOLVED] Magick++ API / OpenMP - special setup? Only one core used

[SOLVED] Magick++ API / OpenMP - special setup? Only one core used

Re: Magick++ API / OpenMP - special setup? Only one core used

Re: Magick++ API / OpenMP - special setup? Only one core used

Re: Magick++ API / OpenMP - special setup? Only one core used

Re: Magick++ API / OpenMP - special setup? Only one core used

Re: Magick++ API / OpenMP - special setup? Only one core used

Re: Magick++ API / OpenMP - special setup? Only one core used

Re: Magick++ API / OpenMP - special setup? Only one core used

Re: Magick++ API / OpenMP - special setup? Only one core used

Re: Magick++ API / OpenMP - special setup? Only one core used

Re: Magick++ API / OpenMP - special setup? Only one core used

Re: Magick++ API / OpenMP - special setup? Only one core used

Re: Magick++ API / OpenMP - special setup? Only one core used

Re: Magick++ API / OpenMP - special setup? Only one core used

Re: Magick++ API / OpenMP - special setup? Only one core used