Page 1 of 1

Using "divide and conquer" for large image processing

Posted: 2011-08-30T12:27:03-07:00
by anyoneis
I have some large tiff files that I am trying to process and I'm looking for performance improvements. Each file has multiple pages, and I am interested in extracting a single page, then breaking it up into 16 sub-images for processing outside of IM. The steps I am currently following are:

1) convert theFile.tif[2] image.mpc

Assuming dimensions of the image are 4W and 4H:
2) convert image.mpc -crop WxH+0+0 +repage -scene 0 -define PNG:color-type-2 theFile_00.png
...
convert image.mpc -crop WxH+3W+3H +repage -scene 0 -define PNG:color-type-2 theFile_15.png

Currently, I am executing two "step 2" processes concurrently, trying to eak out a little more performance.

Does anyone have experience with rearranging things to improve performance? For example, I could try a divide and conquer approach, as in:

1) convert theFile.tiff[2] -crop 4Wx2H +repage imageH%d.mpc // split the image into two

2) convert imageH0.mpc -crop 2Wx2H +repage imageH0Q%d.mpc // split the half into two quarters
convert imageH1.mpc -crop 2Wx2H +repage imageH1Q%d.mpc // split the half into two quarters

3) convert imageH0Q0.mpc -crop 2WxH +repage imageH0Q0O%d.mpc // split the quarter into 2 eights
convert imageH0Q1.mpc -crop 2WxH +repage imageH0Q1O%d.mpc // split the quarter into 2 eights
convert imageH1Q0.mpc -crop 2WxH +repage imageH1Q0O%d.mpc // split the quarter into 2 eights
convert imageH1Q1.mpc -crop 2WxH +repage imageH1Q1O%d.mpc // split the quarter into 2 eights

4) and finally, split each eighth into 16ths, assuming that is the ultimate tile size needed.

Since TANSTAAFL, and there are a lot more operations with this approach over a simple image to 16 tile crop operation, I suspect that I will not see any improvement in wall-clock times unless and until I get to the point where I have an operation at the leaf level that "fits in memory," at which point the improvement could be considerable.

Am I thinking about this problem correctly? Anyone try something like this? any other ideas?

Thanks,
David

Re: Using "divide and conquer" for large image processing

Posted: 2011-08-30T14:07:16-07:00
by fmw42
see http://www.imagemagick.org/Usage/crop/#crop_equal


convert image.tif[0] -crop "4x4@" +repage -define PNG:color-type=2 image_p0_%02d.png

This will give you 16 approx equal sized crop sections.

Note, it is color-type=2 not color-type-2

By default the numbering starts at 0 (or in this case 00) so -scene is not needed

Re: Using "divide and conquer" for large image processing

Posted: 2011-08-30T16:23:49-07:00
by anyoneis
That's a handy beast! I missed that.
convert image.tif[0] -crop "4x4@" +repage -define PNG:color-type=2 image_p0_%02d.png
"%02d" gives me a neater "ls" or "dir"!
By default the numbering starts at 0 (or in this case 00) so -scene is not needed
Just a tiny little "gotcha" there - by default, it starts at the scene you have extracted, which in the case of your example above, is 0, but in my case of extracting scene 2, I need the "scene 0" to reset the value.

Thanks for the hints and comments!
David

Re: Using "divide and conquer" for large image processing

Posted: 2011-08-31T08:53:08-07:00
by anyoneis
This is a naive "pipe dream." As we used to say in the microcomputer industry about "negative incremental return" schemes, "We lose a little bit on each one but we make it up in volume." ;-)

I will be better off just using the "-crop XxY@" syntax pointed out by Fred to produce the output tiles directly from the input file without any intermediate files.

Thanks again!
David