How to optimize rescaling of JPEG 24683x2048

[aleks]

Hi.

We need to create smaller pictures from a bigone.
We call convert with this parameters.

Code: Select all

convert /home/webuser/data/import/512/2016_1228_115000.jpg -strip \
  ( -clone 0 -write /data/tmp_recent/512_2016_1228_115000_full.jpg \
    ( -clone 0 -crop 3085x2048 -set filename:tile %[fx:page.x/3085]_%[fx:page.y/2048] +repage +adjoin -write /data/data2/cams/512/2016/12/28/11-50-00_full_%[filename:tile].jpg -delete 0--1 ) -delete 0--1 ) \
  ( -clone 0 -sample 13016x1080 -write /data/tmp_recent/512_2016_1228_115000_hd.jpg
    ( -clone 0 -crop 3254x1080 -set filename:tile %[fx:page.x/3254]_%[fx:page.y/1080] +repage +adjoin -write /data/data2/cams/512/2016/12/28/11-50-00_hd_%[filename:tile].jpg -delete 0--1 ) -delete 0--1 ) \
  ( -clone 0 -sample 9642x800 -write /data/tmp_recent/512_2016_1228_115000_default.jpg \
    ( -clone 0 -crop 2410x800 -set filename:tile %[fx:page.x/2410]_%[fx:page.y/800] +repage +adjoin -write /data/data2/cams/512/2016/12/28/11-50-00_default_%[filename:tile].jpg -delete 0--1 ) -delete 0--1 ) \
  ( -clone 0 -sample 9642x800 -scale 800x600^ -gravity Center -crop 800x600+0+0 +repage -write /data/data2/cams/512/mobile_800x600.jpg -gravity undefined -delete 0--1 ) \
  ( -clone 0 -sample 9642x800 -scale 160x120^ -gravity Center -crop 160x120+0+0 +repage -write /data/data2/cams/512/mobile_160x120.jpg -gravity undefined -delete 0--1 ) \
  ( -clone 0 -sample 5424x450 -write /data/tmp_recent/512_2016_1228_115000_reduced.jpg -write /data/data2/cams/512/2016/12/28/11-50-00_reduced.jpg -delete 0--1 ) \
  ( -clone 0 -sample 5424x450 -scale 1280x160^ -gravity Center -crop 1280x160+0+0 +repage -write /data/data2/cams/512/mobile_1280x160.jpg -gravity undefined -delete 0--1 ) \
  ( -clone 0 -sample 3616x300 -write /data/tmp_recent/512_2016_1228_115000_optimized.jpg -write /data/data2/cams/512/2016/12/28/11-50-00_optimized.jpg -delete 0--1 ) \
  ( -clone 0 -sample 1808x150 -write /data/tmp_recent/512_2016_1228_115000_small.jpg -write /data/data2/cams/512/2016/12/28/11-50-00_small.jpg -delete 0--1 ) \
  ( -clone 0 -sample 1808x150 -scale 800x75^ -gravity Center -crop 800x75+0+0 +repage -write /data/data2/cams/512/mobile_800x75.jpg -gravity undefined -delete 0--1 ) \
  ( -clone 0 -sample 904x75 -write /data/tmp_recent/512_2016_1228_115000_thumb.jpg -write /data/data2/cams/512/2016/12/28/11-50-00_thumb.jpg -delete 0--1 ) \
  null:

This call uses a lot off cpu and I/O's.
Does anyone can tell us if this is the 'best' usage of convert or is there a better way to do this?

convert -version
Version: ImageMagick 6.6.9-7 2016-06-01 Q16 http://www.imagemagick.org
Copyright: Copyright (C) 1999-2011 ImageMagick Studio LLC
Features: OpenMP

Souce Image:
identify /home/webuser/data/import/512/2016_1228_114000.jpg
/home/webuser/data/import/512/2016_1228_114000.jpg JPEG 24683x2048 24683x2048+0+0 8-bit DirectClass 8.25MB 0.000u 0:00.000

Thanks for any help

[fmw42]

Perhaps you could explain functionally what you are doing. It is hard to follow all your commands, but it seems that you are cropping and resizing. Can you give a bit more detailed outline of your processing?

Repeated cloning and then deleting can be wasteful. You could write your input image once to mpr format and then call that in-memory image over and over. See http://www.imagemagick.org/Usage/files/#mpr

You are cloning and sampling twice in pairs, where the first sample is the same. You could save that clone or write to mpr, so you do not have to do the same sample more than once.

Or why not just use -resize (or -scale) once to the final size for the two processes.

P.S. IM 6.6.9.7 is extremely old (over 250 version old). You might consider an upgrade to the current version.

[aleks]

We have to compose multiple fixed sizes from the source image as well of several dynamically defined compositions such as
resize > then crop > then scale > then add text > scale again > and write result as file XY.jpg.

Since sample performs a lot better than resize and scale we chose to compose sets of:
clone source image > then sample > then do something > then write

We need to sample the source for each chain of clone commands from the unscaled source image.
Deleting without capsulation of clone turned out to be quite error prone.

Yes I know that 6.6.9.7 is old. I haven't seen a newer version for Ubuntu 12.04.5 LTS

[fmw42]

why can you not just crop, then sample and add text to the size you want one time

Have you tried replacing all the -clone 0 ... -delete with -write mpr:xxx +delete of your original and then just use mpr:xxx for each time you want to -clone 0? That saves the repeated cloning and deleting each time.

[aleks]

Thanks for your very valuable input.

We will need to test this.
This will take some time.

I will update the thread as soon as we have tested your suggestions.

[snibgo]

You have fragments like this:

Code: Select all

( -clone 0 -sample 13016x1080 -write fileA.jpg
  ( -clone 0 -crop 3254x1080 +repage +adjoin -write fileB.jpg -delete 0--1 ) 
-delete 0--1 ) \

I can't see why you have the second "-clone". It makes a copy, but you don't need the original any more, so why copy it?

Code: Select all

( -clone 0 -sample 13016x1080 -write fileA.jpg
-crop 3254x1080 +repage +adjoin -write fileB.jpg -delete 0--1 ) \

[fmw42]

You use -gravity undefined, but there is no option "undefined" for -gravity! Did you mean +gravity or -gravity none to reset the gravity setting?

[aleks]

Hi.

we have now change to mpr but haven't faced a better performance.

OLD:

Code: Select all

time convert /web/im/test.jpg -strip \ 
  ( -clone 0 -write /web/im/out/512_2016_1228_115000_full.jpg \ 
    ( -clone 0 -crop 3085x2048 -set filename:tile %[fx:page.x/3085]_%[fx:page.y/2048] +repage +adjoin -write /web/im/out/11-50-00_full_%[filename:tile].jpg -delete 0--1 \) -delete 0--1 \) \ 
  ( -clone 0 -sample 13016x1080 -write /web/im/out/512_2016_1228_115000_hd.jpg \ 
    ( -clone 0 -crop 3254x1080 -set filename:tile %[fx:page.x/3254]_%[fx:page.y/1080] +repage +adjoin -write /web/im/out/11-50-00_hd_%[filename:tile].jpg -delete 0--1 \) -delete 0--1 \) \ 
  ( -clone 0 -sample 9642x800 -write /web/im/out/512_2016_1228_115000_default.jpg \ 
    ( -clone 0 -crop 2410x800 -set filename:tile %[fx:page.x/2410]_%[fx:page.y/800] +repage +adjoin -write /web/im/out/11-50-00_default_%[filename:tile].jpg -delete 0--1 \) -delete 0--1 \) \ 
  ( -clone 0 -sample 9642x800 -scale 800x600^ -gravity Center -crop 800x600+0+0 +repage -write /web/im/out/mobile_800x600.jpg +gravity -delete 0--1 \) \ 
  ( -clone 0 -sample 9642x800 -scale 160x120^ -gravity Center -crop 160x120+0+0 +repage -write /web/im/out/mobile_160x120.jpg +gravity -delete 0--1 \) \ 
  ( -clone 0 -sample 5424x450 -write /web/im/out/512_2016_1228_115000_reduced.jpg -write /web/im/out/11-50-00_reduced.jpg -delete 0--1 \) \ 
  ( -clone 0 -sample 5424x450 -scale 1280x160^ -gravity Center -crop 1280x160+0+0 +repage -write /web/im/out/mobile_1280x160.jpg +gravity -delete 0--1 \) \ 
  ( -clone 0 -sample 3616x300 -write /web/im/out/512_2016_1228_115000_optimized.jpg -write /web/im/out/11-50-00_optimized.jpg -delete 0--1 \) \ 
  ( -clone 0 -sample 1808x150 -write /web/im/out/512_2016_1228_115000_small.jpg -write /web/im/out/11-50-00_small.jpg -delete 0--1 \) \ 
  ( -clone 0 -sample 1808x150 -scale 800x75^ -gravity Center -crop 800x75+0+0 +repage -write /web/im/out/mobile_800x75.jpg +gravity -delete 0--1 \) \ 
  ( -clone 0 -sample 904x75 -write /web/im/out/512_2016_1228_115000_thumb.jpg -write /web/im/out/11-50-00_thumb.jpg -delete 0--1 \) null:

real 0m15.630s
user 0m9.514s
sys 0m6.086s

NEW:

Code: Select all

time convert /web/im/test.jpg -write mpr:panoimage +delete -strip \ 
  ( mpr:panoimage -write /web/im/out/512_2016_1228_115000_full.jpg \ 
    ( mpr:panoimage -crop 3085x2048 -set filename:tile %[fx:page.x/3085]_%[fx:page.y/2048] +repage +adjoin -write /web/im/out/11-50-00_full_%[filename:tile].jpg -delete 0--1 \) -delete 0--1 \) \ 
  ( mpr:panoimage -sample 13016x1080 -write /web/im/out/512_2016_1228_115000_hd.jpg \ 
    ( mpr:panoimage -crop 3254x1080 -set filename:tile %[fx:page.x/3254]_%[fx:page.y/1080] +repage +adjoin -write /web/im/out/11-50-00_hd_%[filename:tile].jpg -delete 0--1 \) -delete 0--1 \) \ 
  ( mpr:panoimage -sample 9642x800 -write /web/im/out/512_2016_1228_115000_default.jpg \
    ( mpr:panoimage -crop 2410x800 -set filename:tile %[fx:page.x/2410]_%[fx:page.y/800] +repage +adjoin -write /web/im/out/11-50-00_default_%[filename:tile].jpg -delete 0--1 \) -delete 0--1 \) \ 
  ( mpr:panoimage -sample 9642x800 -scale 800x600^ -gravity Center -crop 800x600+0+0 +repage -write /web/im/out/mobile_800x600.jpg +gravity -delete 0--1 \) \ 
  ( mpr:panoimage -sample 9642x800 -scale 160x120^ -gravity Center -crop 160x120+0+0 +repage -write /web/im/out/mobile_160x120.jpg +gravity -delete 0--1 \) \ 
  ( mpr:panoimage -sample 5424x450 -write /web/im/out/512_2016_1228_115000_reduced.jpg -write /web/im/out/11-50-00_reduced.jpg -delete 0--1 \) \ 
  ( mpr:panoimage -sample 5424x450 -scale 1280x160^ -gravity Center -crop 1280x160+0+0 +repage -write /web/im/out/mobile_1280x160.jpg +gravity -delete 0--1 \) \
  ( mpr:panoimage -sample 3616x300 -write /web/im/out/512_2016_1228_115000_optimized.jpg -write /web/im/out/11-50-00_optimized.jpg -delete 0--1 \) \ 
  ( mpr:panoimage -sample 1808x150 -write /web/im/out/512_2016_1228_115000_small.jpg -write /web/im/out/11-50-00_small.jpg -delete 0--1 \) \ 
  ( mpr:panoimage -sample 1808x150 -scale 800x75^ -gravity Center -crop 800x75+0+0 +repage -write /web/im/out/mobile_800x75.jpg +gravity -delete 0--1 \) \ 
  ( mpr:panoimage -sample 904x75 -write /web/im/out/512_2016_1228_115000_thumb.jpg -write /web/im/out/11-50-00_thumb.jpg -delete 0--1 \) null:

real 0m29.699s
user 0m17.541s
sys 0m12.131s

We just replaced „\( -clone 0" with „\( mpr:panoimage" to get the picture from mpr instead to clone it.

Mybe everything is okay and the write to the filesystem is the bottleneck.
We just listen to the convert experts if there is a possible optimization.

Many thanks for your help