Export Speed with ImageMagick

[Seth]

I have a repository on GitLab dedicated to emotes, it uses CI to take the original emotes and masks and exports them all in various colors and sizes.

May I ask based on the setup does anyone have a solution that may speed up the build time for these emotes however, even if just by seconds?
The file that actually exports all the images in all sizes and colors:
https://gitlab.com/Elypia/elypia-emotes ... s/build.sh

The original size of emotes are 512px (some are 500px, and some old emotes which are in the process of being redone are 128px).

Currently the repo has 60 emotes, so it does 900 exports in roughly 2.5 minutes. (512, 256, 128)
In the past it was 60 emotes with with more sizes, 3,000 exports in around 4 minutes. (512, 258, 128, 112, 72, 64, 56, 36, 28, 18)

Any input would be handy, whether it's ImageMagick related or not, I just hoping to reduce the overall wait between commit and getting the emotes.

[fmw42]

You can speed it up by writing all sizes from one image in one command line rather than re-reading the input for each size. See https://imagemagick.org/Usage/files/#write

[magick]

By default, ImageMagick supports HDRI which requires floating-point pixel components. If you build ImageMagick without HDRI support (add --disable-HDRI to your configure script command-line), you get unsigned pixels and will likely result in a substantial speed-up.

[fmw42]

You could also build a Q8. That might help also.

[Seth]

You can speed it up by writing all sizes from one image in one command line rather than re-reading the input for each size. See https://imagemagick.org/Usage/files/#write

Thank you for the advice, I managed to get this working with sizes only earlier making a huge difference from 2.5 minutes to 1.5 minutes already.
I'm currently trying to make it work colors as well now but am having a slight problem, may I get your advice?

The below is what the script looks like now:

Code: Select all

#!/bin/bash
if [ -d output/ ]; then rm -rf output/*; fi

# Append here for colors (Color | Prefix | Hue)
declare -A colors
colors[0,0]="red";    colors[0,1]="";  colors[0,2]="100,100,100";
colors[1,0]="blue";   colors[1,1]="b"; colors[1,2]="100,100,0";
colors[2,0]="green";  colors[2,1]="g"; colors[2,2]="100,100,166";
colors[3,0]="pink";   colors[3,1]="p"; colors[3,2]="100,100,66.6";
colors[4,0]="yellow"; colors[4,1]="y"; colors[4,2]="100,115,115";

for c in 0 1 2 3 4
do
    export_arg="${export_arg} \( +clone -modulate ${colors[${c},2]} +clip-mask"

    # When adding a new resolution, just append the output size here.
    for size in 512 258 128 112 72 64 56 36 28 18
    do
        mkdir -p output/${colors[${c},0]}/${size}px/
        export_arg="${export_arg} \( +clone -resize ${size}X${size} -write \"output/${colors[${c},0]}/${size}px/${colors[${c},1]}[EMOTE]\" +delete \)"
    done

    export_arg="${export_arg} +delete \)"
done

echo ${export_arg}

for file in emotes/*
do
    title=$(identify -format "%t" ${file})
    emote="${title}.png"
    mask="masks/${title}Mask.png"

    if [ -f ${mask} ]; then
        mask_arg="-clip-mask ${mask}"
    else
        unset mask_arg
    fi

    echo "Exporting: ${file}"

    emote_export_arg=${export_arg//\[EMOTE\]/${emote}}
    echo "magick ${file} -filter Catrom ${mask_arg} ${emote_export_arg} null:"
    magick ${file} -filter Catrom ${mask_arg} ${emote_export_arg} null:
done

Code: Select all

magick emotes/pandaWut.png -filter Catrom   \( +clone -modulate 100,100,100 +clip-mask \( +clone -resize 512X512 -write "output/red/512px/pandaWut.png" +delete \) \( +clone -resize 258X258 -write "output/red/258px/pandaWut.png" +delete \) \( +clone -resize 128X128 -write "output/red/128px/pandaWut.png" +delete \) \( +clone -resize 112X112 -write "output/red/112px/pandaWut.png" +delete \) \( +clone -resize 72X72 -write "output/red/72px/pandaWut.png" +delete \) \( +clone -resize 64X64 -write "output/red/64px/pandaWut.png" +delete \) \( +clone -resize 56X56 -write "output/red/56px/pandaWut.png" +delete \) \( +clone -resize 36X36 -write "output/red/36px/pandaWut.png" +delete \) \( +clone -resize 28X28 -write "output/red/28px/pandaWut.png" +delete \) \( +clone -resize 18X18 -write "output/red/18px/pandaWut.png" +delete \) +delete \) \( +clone -modulate 100,100,0 +clip-mask \( +clone -resize 512X512 -write "output/blue/512px/bpandaWut.png" +delete \) \( +clone -resize 258X258 -write "output/blue/258px/bpandaWut.png" +delete \) \( +clone -resize 128X128 -write "output/blue/128px/bpandaWut.png" +delete \) \( +clone -resize 112X112 -write "output/blue/112px/bpandaWut.png" +delete \) \( +clone -resize 72X72 -write "output/blue/72px/bpandaWut.png" +delete \) \( +clone -resize 64X64 -write "output/blue/64px/bpandaWut.png" +delete \) \( +clone -resize 56X56 -write "output/blue/56px/bpandaWut.png" +delete \) \( +clone -resize 36X36 -write "output/blue/36px/bpandaWut.png" +delete \) \( +clone -resize 28X28 -write "output/blue/28px/bpandaWut.png" +delete \) \( +clone -resize 18X18 -write "output/blue/18px/bpandaWut.png" +delete \) +delete \) \( +clone -modulate 100,100,166 +clip-mask \( +clone -resize 512X512 -write "output/green/512px/gpandaWut.png" +delete \) \( +clone -resize 258X258 -write "output/green/258px/gpandaWut.png" +delete \) \( +clone -resize 128X128 -write "output/green/128px/gpandaWut.png" +delete \) \( +clone -resize 112X112 -write "output/green/112px/gpandaWut.png" +delete \) \( +clone -resize 72X72 -write "output/green/72px/gpandaWut.png" +delete \) \( +clone -resize 64X64 -write "output/green/64px/gpandaWut.png" +delete \) \( +clone -resize 56X56 -write "output/green/56px/gpandaWut.png" +delete \) \( +clone -resize 36X36 -write "output/green/36px/gpandaWut.png" +delete \) \( +clone -resize 28X28 -write "output/green/28px/gpandaWut.png" +delete \) \( +clone -resize 18X18 -write "output/green/18px/gpandaWut.png" +delete \) +delete \) \( +clone -modulate 100,100,66.6 +clip-mask \( +clone -resize 512X512 -write "output/pink/512px/ppandaWut.png" +delete \) \( +clone -resize 258X258 -write "output/pink/258px/ppandaWut.png" +delete \) \( +clone -resize 128X128 -write "output/pink/128px/ppandaWut.png" +delete \) \( +clone -resize 112X112 -write "output/pink/112px/ppandaWut.png" +delete \) \( +clone -resize 72X72 -write "output/pink/72px/ppandaWut.png" +delete \) \( +clone -resize 64X64 -write "output/pink/64px/ppandaWut.png" +delete \) \( +clone -resize 56X56 -write "output/pink/56px/ppandaWut.png" +delete \) \( +clone -resize 36X36 -write "output/pink/36px/ppandaWut.png" +delete \) \( +clone -resize 28X28 -write "output/pink/28px/ppandaWut.png" +delete \) \( +clone -resize 18X18 -write "output/pink/18px/ppandaWut.png" +delete \) +delete \) \( +clone -modulate 100,115,115 +clip-mask \( +clone -resize 512X512 -write "output/yellow/512px/ypandaWut.png" +delete \) \( +clone -resize 258X258 -write "output/yellow/258px/ypandaWut.png" +delete \) \( +clone -resize 128X128 -write "output/yellow/128px/ypandaWut.png" +delete \) \( +clone -resize 112X112 -write "output/yellow/112px/ypandaWut.png" +delete \) \( +clone -resize 72X72 -write "output/yellow/72px/ypandaWut.png" +delete \) \( +clone -resize 64X64 -write "output/yellow/64px/ypandaWut.png" +delete \) \( +clone -resize 56X56 -write "output/yellow/56px/ypandaWut.png" +delete \) \( +clone -resize 36X36 -write "output/yellow/36px/ypandaWut.png" +delete \) \( +clone -resize 28X28 -write "output/yellow/28px/ypandaWut.png" +delete \) \( +clone -resize 18X18 -write "output/yellow/18px/ypandaWut.png" +delete \) +delete \) null:

Quite long, but this is the actual command being performed per emote.

However it keeps returning the error:

Code: Select all

magick: unable to open image '\(': No such file or directory @ error/blob.c/OpenBlob/3497.
magick: no decode delegate for this image format `' @ error/constitute.c/ReadImage/556.

However I'm unsure what exactly the syntax I should be going for is regarding nested clones.

[snibgo]

magick: unable to open image '\(':

"magick" should not see the backslash. In normal use it would be stripped by the bash interpreter, but not when it is inside a variable.

I suggest you read how and when bash processes escapes, and expands variables. I think the problem is that you escape the parentheses when setting export_arg, but shouldn't.

[Seth]

magick: unable to open image '\(':

"magick" should not see the backslash. In normal use it would be stripped by the bash interpreter, but not when it is inside a variable.

I suggest you read how and when bash processes escapes, and expands variables. I think the problem is that you escape the parentheses when setting export_arg, but shouldn't.

Thank you very much! You're right, that was the problem.
It's been addressed now and works perfectly!