Imagick and threads

IMagick is a native PHP extension to create and modify images using the ImageMagick API. ImageMagick Studio LLC did not write nor does it maintain the IMagick extension, however, IMagick users are welcome to discuss the extension here.
Post Reply
Prosperent
Posts: 5
Joined: 2011-09-29T10:01:56-07:00
Authentication code: 8675308

Imagick and threads

Post by Prosperent »

First some background. We resize about 50 million source images into 5 different sizes. For this we use a cluster of 16 dual 8 core servers. We don't have imagemagick installed on any of the servers, but rather just imagick through the yum pecl package (yum install php-pecl-imagick). This works fine for us, but each process uses several cpu cores which is a problem because of the level of concurrency we push through the system. I've seen the threads about disabling openMP, but that appears to apply to imagemagick and not php-pecl-imagick. Is there anything I can do to disable threading or set the threads to one with the imagick pecl package?
Prosperent
Posts: 5
Joined: 2011-09-29T10:01:56-07:00
Authentication code: 8675308

Re: Imagick and threads

Post by Prosperent »

I found /usr/lib64/ImageMagick-6.6.5/config which has the policy file and such. Does the pecl package just bundle the ImageMagick library with it? Would be my guess since I can't issue convert -version at the command line. So, is there a way to recompile the version that it installs with the pecl package, or is there a way at runtime to change to one thread instead of what it appears to have defaulted to (16)
User avatar
magick
Site Admin
Posts: 11064
Joined: 2003-05-31T11:32:55-07:00

Re: Imagick and threads

Post by magick »

Try adding
  • <policy domain="resource" name="thread" value="1"/>
to your policy.xml file. That sets the maximum threads to 1 site-wide. Alternatively you can set the MAGICK_THREAD_LIMIT environment variable to set the maximum number of threads for a particular ImageMagick script.
Prosperent
Posts: 5
Joined: 2011-09-29T10:01:56-07:00
Authentication code: 8675308

Re: Imagick and threads

Post by Prosperent »

Wanted to come back and update. We pushed this change and the resulting server load decrease was dramatic. We went from a cluster wide average load of 40 to an average load of 4. We can literally get rid of half of our servers after this single change. Our image processing tasks used to spike to 300-500 percent cpu usage per process, and now they are almost always less than 10.
Post Reply