Page 1 of 1
Resizing really high res images hangs Drupal
Posted: 2018-05-28T12:24:10-07:00
by k6pski
Hi!
I'm co-maintaining this Drupal module
https://www.drupal.org/project/batch_resize_image and I'm having trouble getting all of our 10 000+ images resized. In the background it's using the ImageMagic module
https://www.drupal.org/project/imagemagick
At some point the script will just hang. I think I've now come to conclusion that it fails with really high resolution images like this:
Code: Select all
identify das_str_2.png
das_str_2.png PNG 16535x8184 16535x8184+0+0 8-bit DirectClass 4.666MB 0.000u 0:00.000
Locally on my Macbook Pro it seems ok but the images need to eventually resized on Amazon Beanstalk instance and it has little bit less resources. So while the php stops at the big images and just doesn't do anything then manually running the command takes almost 4 minutes but finishes:
Code: Select all
time convert das_str_2.png -resize 5000x2475! -quality 75 das_str_2-2.png
real 3m40.560s
user 0m20.804s
sys 0m1.184s
The version of convert is:
Code: Select all
convert --version
Version: ImageMagick 6.7.8-9 2016-06-22 Q16 http://www.imagemagick.org
Copyright: Copyright (C) 1999-2012 ImageMagick Studio LLC
Features: OpenMP
There's about 2GB ram on the system.
This is the image I'm talking about
https://www.upload.ee/image/8493734/das_str_2.png
I'm running out of ideas. How should I solve this? Is it problem from php side if the convert runs ok on bash?
Thank you!
edit:
I've made error in my calculation as the image is converted on a network drive mount.
Code: Select all
time convert das_str_2.png -resize 5000x2475! -quality 75 das_str_2-2.png
real 5m3.598s
user 0m21.407s
sys 0m1.307s
Re: Resizing really high res images hangs Drupal
Posted: 2018-05-28T12:46:36-07:00
by Bonzo
php has a standard time limit of 30 seconds. Normally you can adjust it in the php.ini file but that may not be possible on Amazon.
I assume as Imagemagick is running you must have "Safe mode" disabled so you could look at ini_set() and set_time_limit() but these options may be disabled by Amazon.
Re: Resizing really high res images hangs Drupal
Posted: 2018-05-28T12:53:07-07:00
by k6pski
I can change the settings but shouldn't it just die with timeout error then? I'll check tomorrow.
Also the image convert takes longer as I didn't run the convert on the network mount as it should. It takes 5 minutes to convert that image.
Re: Resizing really high res images hangs Drupal
Posted: 2018-05-28T13:20:42-07:00
by fmw42
Check your tmp directory to be sure it is not filling up or filled up with left over imagemagick files.
Re: Resizing really high res images hangs Drupal
Posted: 2018-05-28T22:36:45-07:00
by k6pski
I just killed the instance with this 12 000 x 12 000 image
https://wallportal.com/uploads/posts/ea ... on_026.jpg
With this 12858x6534 image it took only 12 seconds to resize
http://hanedanrpg.com/photos/hanedanrpg/12/55931.jpg
And my problematic image linked in the first post which is 16535x8184 takes 3-5 minutes to resize.
Code: Select all
12000x12000 = 144000000 killed server
12858x6534 = 84014172 resize in 12 seconds
16535x8184 = 135322440 resize in 3 minutes
Server doesn't have swap.
edit: sry, it didn't kill the server:
Code: Select all
time convert earth_wallpaper_high_resolution_026.jpg -resize 5000x2475! -quality 75 earth_wallpaper_high_resolution_026-2.jpg
Killed
real 24m49.255s
user 0m13.913s
sys 0m0.767s
Re: Resizing really high res images hangs Drupal
Posted: 2018-05-28T22:45:21-07:00
by k6pski
fmw42 wrote: ↑2018-05-28T13:20:42-07:00
Check your tmp directory to be sure it is not filling up or filled up with left over imagemagick files.
Thre's one tmp file
Code: Select all
ls -lah /tmp/magick-*
-rw------- 1 root root 95M May 29 05:33 /tmp/magick-hZidmMde
Re: Resizing really high res images hangs Drupal
Posted: 2018-05-28T23:24:32-07:00
by fmw42
You have a 95 MB temp file there from ImageMagick. You can delete it to save space.
Re: Resizing really high res images hangs Drupal
Posted: 2018-05-28T23:55:36-07:00
by k6pski
Thank you!
I think I also found the problem. When we increased the ram to 4GB the problematic image was resized within 12 seconds instead of 3-5 minutes. Will do more tests.
Re: Resizing really high res images hangs Drupal
Posted: 2018-06-04T16:46:50-07:00
by fmw42
You cannot upload images to this forum. Please upload to some free hosting service and put the URL(s) here. Also please always provide your ImageMagick version and platform and your command line or code.
Often resizing an image to smaller dimensions creates large file sizes when the input is type palette (8-bit per pixel). The resize adds new colors that are more that 256 colors. Thus the output file size must be 24-bit or 32-bit per pixel. So you have to reduce colors again. You may need to use +dither -colors 256 and/or add -type palette or if PNG add PNG8: to your output image as PNG8:ouput.png
Re: Resizing really high res images hangs Drupal
Posted: 2018-06-04T18:11:42-07:00
by snibgo
[I've removed spam from the thread.]
