I have some large tiff files that I am trying to process and I'm looking for performance improvements. Each file has multiple pages, and I am interested in extracting a single page, then breaking it up into 16 sub-images for processing outside of IM. The steps I am currently following are:
1) convert theFile.tif[2] image.mpc
Assuming dimensions of the image are 4W and 4H:
2) convert image.mpc -crop WxH+0+0 +repage -scene 0 -define PNG:color-type-2 theFile_00.png
...
convert image.mpc -crop WxH+3W+3H +repage -scene 0 -define PNG:color-type-2 theFile_15.png
Currently, I am executing two "step 2" processes concurrently, trying to eak out a little more performance.
Does anyone have experience with rearranging things to improve performance? For example, I could try a divide and conquer approach, as in:
1) convert theFile.tiff[2] -crop 4Wx2H +repage imageH%d.mpc // split the image into two
2) convert imageH0.mpc -crop 2Wx2H +repage imageH0Q%d.mpc // split the half into two quarters
convert imageH1.mpc -crop 2Wx2H +repage imageH1Q%d.mpc // split the half into two quarters
3) convert imageH0Q0.mpc -crop 2WxH +repage imageH0Q0O%d.mpc // split the quarter into 2 eights
convert imageH0Q1.mpc -crop 2WxH +repage imageH0Q1O%d.mpc // split the quarter into 2 eights
convert imageH1Q0.mpc -crop 2WxH +repage imageH1Q0O%d.mpc // split the quarter into 2 eights
convert imageH1Q1.mpc -crop 2WxH +repage imageH1Q1O%d.mpc // split the quarter into 2 eights
4) and finally, split each eighth into 16ths, assuming that is the ultimate tile size needed.
Since TANSTAAFL, and there are a lot more operations with this approach over a simple image to 16 tile crop operation, I suspect that I will not see any improvement in wall-clock times unless and until I get to the point where I have an operation at the leaf level that "fits in memory," at which point the improvement could be considerable.
Am I thinking about this problem correctly? Anyone try something like this? any other ideas?
Thanks,
David
Using "divide and conquer" for large image processing
- fmw42
- Posts: 25562
- Joined: 2007-07-02T17:14:51-07:00
- Authentication code: 1152
- Location: Sunnyvale, California, USA
Re: Using "divide and conquer" for large image processing
see http://www.imagemagick.org/Usage/crop/#crop_equal
convert image.tif[0] -crop "4x4@" +repage -define PNG:color-type=2 image_p0_%02d.png
This will give you 16 approx equal sized crop sections.
Note, it is color-type=2 not color-type-2
By default the numbering starts at 0 (or in this case 00) so -scene is not needed
convert image.tif[0] -crop "4x4@" +repage -define PNG:color-type=2 image_p0_%02d.png
This will give you 16 approx equal sized crop sections.
Note, it is color-type=2 not color-type-2
By default the numbering starts at 0 (or in this case 00) so -scene is not needed
Re: Using "divide and conquer" for large image processing
That's a handy beast! I missed that.
"%02d" gives me a neater "ls" or "dir"!convert image.tif[0] -crop "4x4@" +repage -define PNG:color-type=2 image_p0_%02d.png
Just a tiny little "gotcha" there - by default, it starts at the scene you have extracted, which in the case of your example above, is 0, but in my case of extracting scene 2, I need the "scene 0" to reset the value.By default the numbering starts at 0 (or in this case 00) so -scene is not needed
Thanks for the hints and comments!
David
Re: Using "divide and conquer" for large image processing
This is a naive "pipe dream." As we used to say in the microcomputer industry about "negative incremental return" schemes, "We lose a little bit on each one but we make it up in volume."
I will be better off just using the "-crop XxY@" syntax pointed out by Fred to produce the output tiles directly from the input file without any intermediate files.
Thanks again!
David
I will be better off just using the "-crop XxY@" syntax pointed out by Fred to produce the output tiles directly from the input file without any intermediate files.
Thanks again!
David