Page 1 of 1

Processing a big batch of files

Posted: 2009-06-16T05:42:54-07:00
by pitosalas
I need to process a directory with hundreds or thousands of very similar files - voting ballots.

I've written the algorithm for the one-at-a-time process - and it's a bit slow for my taste; I am still tuning it.

I am writing in Ruby, with RMagick. My code right now basically is Ruby loop which starts by calling an ImageMagick 'open', does a bunch of ImageMagick calls and produces the results, when it looks back to the top. The images are never moved out of ImageMagick space or processed in any other way than via ImageMagick/Rmagick calls.

The files are .tif, about 160K, 2562 x 4265, gray scale.

Is there a way to speed up the process applying a standard algorithm to a large collection files?

Thanks,

Pito

Re: Processing a big batch of files

Posted: 2009-06-16T07:00:35-07:00
by magick
Our only suggestion is to force the gargage collector to run to free the consumed memory for each iteration. Use Google to find this recommendation out there on the Net somewhere.