batch processing / response file support

[00bins]

Hi

We use imagemagick to process 1000's of images as part of our build process.

As a test I've timed processing a subset of these files - around 1000.

Invoking convert for every file takes around 3 minutes, where each operation is like this

convert -resize 64x64! Character_packs.bmp -type truecolor tmp/Character_packs.bmp

I then have a modified version of convert that I can pass a file with the same operations like below

-resize 64x64! Character_packs.bmp -type truecolor tmp/Character_packs.bmp

and which loops over the file 'creating' argc & argv from the file without having the overhead of invoking convert for everyfile

timing both methods shows that the second method is way faster

$ time sh callperfile.sh

real 2m59.130s
user 2m36.229s
sys 0m19.217s

$ time convert @tmp.txt

real 0m22.214s
user 0m20.685s
sys 0m1.508s

I ran these tests on a mac but the same speed up applies to windows.

I'd like to contribute this change back to the source for imagemagick so that we don't have to keep our own version,
and so that anyone else can make use of this.

How do I go about proposing this change, and are there any docs for coding standards I should adhere to?


cheers
Rob

[fmw42]

I cannot address the issue of contribution.

But I would point out that correct IM 6 and IM 7 processing should have the input should come before the -resize operation. IM syntax is not like Unix command syntax. See https://imagemagick.org/Usage/basics/#why. This is even more important for IM 7, which is much more strict in syntax usage. In IM 6 it may not matter, but it will in IM 7.

[snibgo]

I then have a modified version of convert ...

You can do the job in single command, eg:

Code: Select all

convert \
in1.bmp -resize 64x64! type truecolor -write out1.bmp +delete \
in2.bmp -resize 64x64! type truecolor -write out2.bmp +delete \
in3.bmp -resize 64x64! type truecolor -write out3.bmp +delete \
:
in999.bmp -resize 64x64! type truecolor -write out999.bmp +delete \
xc: NULL:

Or you can use IM's script facility.

Code: Select all

convert @myscript.txt xc: NULL:

(Note: IM v6 scripting is different to v7.)

[00bins]

Thanks for the replies

re syntax - I'll fix that!

From what I can tell IM scripts don't really address my issue - they are shell/bat/perl/etc.. scripts that still end up invoking magick once per file - unless I'm missing something.

On windows there is a command line length limit of 8k chars. My test with only 1000 files is already 256k chars so I can't use that.

also the example

convert @myscript.txt xc: NULL:

just prints out the usage - and afaict there is nothing in the existing code that knows how to process an argument preceeded by @ (because I had to add that myself) - or am I missing something here too?

thanks
Rob

[00bins]

aha - I've just noticed that magick has a -script option, which would appear to be what I need after all

thanks
Rob

[00bins]

ugh - it's early but I can't seem to find anything on the forums relating to the -script option. Searching for "-script" returns no hits - not even my post above, and no amount of shell-escaping or quoting seems to help.

I've made a small file with what I think should work, but it doesn't do what I want, and I probably should be posting in the users section now anyway

cheers
Rob

[snibgo]

On windows there is a command line length limit of 8k chars.

Yes. The IM script facility has no limit, as far as I can see. I have tested it with long scripts, eg 100KB.