subimage search with transparency

[aurelien.baudet]

Hello,

I have a simple script that searches for the position of an image into another and it works well.
However, if the searched image has transparency, it doesn't find it only in some cases. I don't understand why sometimes it doesn't work.

My script (where w, h, x and y are provided to limit the search):

Code: Select all

convert ^
          +repage ^
          -extract "<w>x<h>+<x>+<y>" "<source>" "<tempSource>"

compare ^
   -metric "rmse" -subimage-search -dissimilarity-threshold "0.1" ^
  -virtual-pixel "edge" "<tempSource>" "<search image>" "<compare result>"

• It works with following source (works in png or jpg)
  
  and with the following search image
  
  It find it at position 516,514
• It works with following source (works in png or jpg)
  
  and with the following search image (no transparency)
  
  It find it at position 326,71
• But with the following source works in jpg but not in png
  
  and the following search image (with transparency) it doesn't work
  
  It doesn't find it

Thanks for your help

[fmw42]

The only compare metric that takes transparency into account is -fuzz. See http://www.imagemagick.org/Usage/compare/#statistics

You may need to flatten the two images against the same background and do the compare that way if you want to use other metrics.

Note, doing compare properly taking transparency into account has been requested. But not yet implemented.

[aurelien.baudet]

Hello,

I have tried with fuzz metric, set same background color and both. However, this doesn't work with the 2 last images (in png).

Someone has a solution to make it work ?

[fmw42]

JPG does not support transparency. It turns it off. So the background is whatever you have under the transparency.

[anthony]

I have had a similar problem with matching up parts of incomplete but overlaping maps.

See discussion.
viewtopic.php?f=1&t=22526&p=95286&hilit=overlap#p95286

The program and technique actually does not care if the search image is overlapping or not, but it did have to deal with the transparency problem.

To avoid the 'transparency' issue, I grabbed sub-sections of the search image that had no transparency (using dilate to mask sub-image selection). Also to prevent selection of a sub-image that contained no useful search pattern, I also selected the sub-images by centering them on areas of high-entropy (lots of color changes).

The selection is skewed to actually look for the sub-images around the edges before trying one in the middle, (next sub-image is the next valid center point that is furthest from all other sub-image attempts so far) but that is because the focus was on 'overlaps' but it really did not care if the result was an overlap or fully contained.

NOTE original and real reason for using small sub-images for the search was not transparency (though it solved that issue), but for speed in searching the larger image for possible locations of the new 'overlapping' image, the exact same technique (and program) should work for you in finding the location of the 'transparent' shaped image on the larger image.

The script
http://www.imagemagick.org/Usage/scripts/overlap
currently is set up to continually find matches until 3 sub-images match the same location, and to abort (no match) if we get 3 different matches! Otherwise it aborts if the distance between sub-images is getting too small (that can take a long time).

Basically it means it needs to find 3 sub-images of the search images that matches to same 'overlay' location. While a 'match' is simply a threshold of the default output of "compare".

These are hard corded in the script for my purposes, but the script is very well commented and you can adjust as you like, or generate your own, now that the technique has been worked out.

[anthony]

In summary...

find a sub-image that has no transparency, and search using that.

A Distance morphology on the extracted alpha mask of the search image, can find the largest 'square' (or with some more work rectangle) that has no transparency.

However I still advocate finding smaller "high entropy, with no transparency" sub-images for the search, to find 'possible match' point(s).

"Compare" however does still needs a good 'masked' compare method even if it is not a 'sub-image' compare, at least as a better way of checking if, the sub-image 'possible match' is actually a proper fill search-image 'match' while ignoring transparency in at least the search-image.

Optionally ignore transparent pixels in both search-image and the larger image-to-search, if there is a possiblity of 'partial overlaps'

That needs two output results... how good the match is, and how many pixels 'overlaped'. (it is no good having a perfect match if there is only a one pixel overlap).

[saif khan]

a