Using "divide and conquer" for large image processing
Posted: 2011-08-30T12:27:03-07:00
I have some large tiff files that I am trying to process and I'm looking for performance improvements. Each file has multiple pages, and I am interested in extracting a single page, then breaking it up into 16 sub-images for processing outside of IM. The steps I am currently following are:
1) convert theFile.tif[2] image.mpc
Assuming dimensions of the image are 4W and 4H:
2) convert image.mpc -crop WxH+0+0 +repage -scene 0 -define PNG:color-type-2 theFile_00.png
...
convert image.mpc -crop WxH+3W+3H +repage -scene 0 -define PNG:color-type-2 theFile_15.png
Currently, I am executing two "step 2" processes concurrently, trying to eak out a little more performance.
Does anyone have experience with rearranging things to improve performance? For example, I could try a divide and conquer approach, as in:
1) convert theFile.tiff[2] -crop 4Wx2H +repage imageH%d.mpc // split the image into two
2) convert imageH0.mpc -crop 2Wx2H +repage imageH0Q%d.mpc // split the half into two quarters
convert imageH1.mpc -crop 2Wx2H +repage imageH1Q%d.mpc // split the half into two quarters
3) convert imageH0Q0.mpc -crop 2WxH +repage imageH0Q0O%d.mpc // split the quarter into 2 eights
convert imageH0Q1.mpc -crop 2WxH +repage imageH0Q1O%d.mpc // split the quarter into 2 eights
convert imageH1Q0.mpc -crop 2WxH +repage imageH1Q0O%d.mpc // split the quarter into 2 eights
convert imageH1Q1.mpc -crop 2WxH +repage imageH1Q1O%d.mpc // split the quarter into 2 eights
4) and finally, split each eighth into 16ths, assuming that is the ultimate tile size needed.
Since TANSTAAFL, and there are a lot more operations with this approach over a simple image to 16 tile crop operation, I suspect that I will not see any improvement in wall-clock times unless and until I get to the point where I have an operation at the leaf level that "fits in memory," at which point the improvement could be considerable.
Am I thinking about this problem correctly? Anyone try something like this? any other ideas?
Thanks,
David
1) convert theFile.tif[2] image.mpc
Assuming dimensions of the image are 4W and 4H:
2) convert image.mpc -crop WxH+0+0 +repage -scene 0 -define PNG:color-type-2 theFile_00.png
...
convert image.mpc -crop WxH+3W+3H +repage -scene 0 -define PNG:color-type-2 theFile_15.png
Currently, I am executing two "step 2" processes concurrently, trying to eak out a little more performance.
Does anyone have experience with rearranging things to improve performance? For example, I could try a divide and conquer approach, as in:
1) convert theFile.tiff[2] -crop 4Wx2H +repage imageH%d.mpc // split the image into two
2) convert imageH0.mpc -crop 2Wx2H +repage imageH0Q%d.mpc // split the half into two quarters
convert imageH1.mpc -crop 2Wx2H +repage imageH1Q%d.mpc // split the half into two quarters
3) convert imageH0Q0.mpc -crop 2WxH +repage imageH0Q0O%d.mpc // split the quarter into 2 eights
convert imageH0Q1.mpc -crop 2WxH +repage imageH0Q1O%d.mpc // split the quarter into 2 eights
convert imageH1Q0.mpc -crop 2WxH +repage imageH1Q0O%d.mpc // split the quarter into 2 eights
convert imageH1Q1.mpc -crop 2WxH +repage imageH1Q1O%d.mpc // split the quarter into 2 eights
4) and finally, split each eighth into 16ths, assuming that is the ultimate tile size needed.
Since TANSTAAFL, and there are a lot more operations with this approach over a simple image to 16 tile crop operation, I suspect that I will not see any improvement in wall-clock times unless and until I get to the point where I have an operation at the leaf level that "fits in memory," at which point the improvement could be considerable.
Am I thinking about this problem correctly? Anyone try something like this? any other ideas?
Thanks,
David