[howto] running batch jobs in parallel on Windows

[jftuga]

I have created a program that runs a command-line executable in parallel on a group of similar files and thought this would be useful to Windows ImageMagick users. I was able to run the convert.exe program to convert 1440 jpg files (avg size: 135kb) to tiff files. On a 16 CPU system this created 1440 tiff files in 74 seconds, which is just under 20 image conversions per second.

My program is called mp, short for multi-process. From a command line, you would use:

mp.exe convert “_input_ _base_.tiff” -- *.jpg

Anything inside of underscores are macros that get substituted with file names. See the website for more information.
The program figures out the number of CPUs your system has and runs that many subprocesses concurrently.

1) https://github.com/jftuga/Windows/tree/master/mp
2) click on the mp.exe link
3) click on Raw to download the actual mp.exe exectuable
4) you can also download the AutoIt source code by clicking on mp.au3

If you decide to try this out, please reply as I love to get some feedback.

Thanks,
-John

[anthony]

Have you looked at "parellel" in GNU tools for UNIX?
http://www.gnu.org/software/parallel/

Check its manual. It has a incredible range of capabilities, including its use as a direct drop in replacement for xargs.

It may be compilable under windows. But as I am not a UNIX user, I don't know.

[jftuga]

I did see this, but then saw this thread which says the program is not supported under Windows, but still may run under cygwin.
http://lists.gnu.org/archive/html/paral ... 00008.html

It has been awhile since I used Linux on a day-to-day basis. However, I do use natively compiled versions of awk, sed, grep, etc. under Windows regularly.

-John