Compare (subimage-search) rotated image without intermediary files

[snibgo]

You show "An example of a cropped subimage". But the background should be transparent, correct? And you want your script to identify the last two as matches?

Then forget my optimization step 3. It won't work.

Could both the matching icons be cropped?

Your comparison method needs to treat transparent pixels as "wildcard", so they match any colour in the other image. Either that, or mask both images to the non-transparent pixels of the other before comparing.

[cowboyzed]

Could both the matching icons be cropped?

The first match may be cropped, but not more than 5px. The second match could be cropped as I show in the image above.

Your comparison method needs to treat transparent pixels as "wildcard", so they match any colour in the other image. Either that, or mask both images to the non-transparent pixels of the other before comparing.

I don't understand what this will change in the command to use.

[fmw42]

In IM 7, you can do a compare and make it insensitive to transparent pixels, so it only matches opaque pixels. see viewtopic.php?f=4&t=31053

Can you show one example of your smaller reference image? I am still confused about what you are using to compare against the larger image.

[cowboyzed]

The last example gives :
then

untill

(this last is the matching icon)

[fmw42]

Please be more specific with your replies. What last example? Can you share your exact command line and the two input images? That way we can try to reproduce your results.

P.S. Note my request at viewtopic.php?f=2&t=31938

[cowboyzed]

Please note the algorithm proposed by snigbo : https://www.imagemagick.org/discourse-s ... 28#p146096

With the big image :



We have to detect this subimage which is repeated :



PS: the unique input is the big image.

[fmw42]

You have potentially 3 issues that you need to fix with your images.

1) You have cropped the small image without +repage, so that it has a virtual canvas. This messes up the rotation.

2) You have texture under the transparent areas. This needs to made into a constant background color

3) You may want to pad your large image with more transparency to account for rotations of your small image at angles other than increments of 90 deg. For example at 45 deg rotation, the small image will increase in size by a factor of 1.414. Without padding the large image, it may prevent the rotated small image from being moved to a location that properly matches its corresponding object in the larger image. For the example below, I have only rotated the small image 180 deg, so its size does not change.


The following works fine for me on IM 6.9.8.5 Q16 Mac OSX

Code: Select all

convert \
\( 370669patch.png +repage -background black -alpha background \) \
\( 224977object4.png +repage -background black -alpha background \
    -background none -virtual-pixel background -distort SRT 180 -trim +repage \) miff:- |\
compare -metric rmse -subimage-search -dissimilarity-threshold 1 - diff.png

4734.52 (0.0722441) @ 269,40





Likewise using -rotate rather than -distort SRT

Code: Select all

convert \
\( 370669patch.png +repage -background black -alpha background \) \
\( 224977object4.png +repage -background black -alpha background \
    -background none -rotate 180 -trim +repage \) miff:- |\
compare -metric rmse -subimage-search -dissimilarity-threshold 1 - diff.png

4808.31 (0.0733701) @ 269,40


Likewise using +distort SRT

Code: Select all

convert \
\( 370669patch.png +repage -background black -alpha background \) \
\( 224977object4.png +repage -background black -alpha background \
    -background none -virtual-pixel background +distort SRT 180 -trim +repage \) miff:- |\
compare -metric rmse -subimage-search -dissimilarity-threshold 1 - diff.png

4893.59 (0.0746713) @ 268,39

-distort SRT will make the output the same size as the input, potentially cropping off some of it at angles other than 90 increments. -rotate and +distort SRT will expand the size of the output so that nothing is cropped off. We add a -trim to be sure we do not pad too much during the rotation, though it likely does not matter since your reference (small) image seems to be cropped to its bounding box anyway.

[cowboyzed]

Thank you very much. Your command is useful : I can rotate the small image and use subimage-search without using intermediate files.
I want to get the score matching and I will use the following :

Code: Select all

convert \
\( 370669patch.png +repage -background black -alpha background \) \
\( 224977object4.png +repage -background black -alpha background \
    -background none -virtual-pixel background -distort SRT 180 -trim +repage \) miff:- |\
compare -metric rmse -subimage-search -dissimilarity-threshold 1 - null: 2>&1 

Since I have to loop the command over 10 to 350 degrees, I'm wondering if there is a way to improve the solution you proposed here (https://www.imagemagick.org/discourse-s ... 28#p146091). I think it may increase performance, according to the algorithm proposed by snigbo.

[fmw42]

I do not know if it can be optimized further. But see viewtopic.php?f=2&t=31938#p146190. Once this is fixed, you can do it all in-line in the compare command without any pipes or saved files.

So this should be available in the next release.

[cowboyzed]

Ok. I want to say there is now two differents algorithms :

1. Chop into 6 images, looping over these 6 images, using subimage-search (with rotation) over the big image.

2. Chop into 6 images, comparing each of the 6 images (with rotation) against the others and take the best score.

Now we got the solution of the method 1, but I want to know if the method 2 is better (probably with the appropriate command).

[snibgo]

Of those two algorithms, #1 will take far longer than #2, and most of the effort will be wasted.

When doing the rotations, you could go in 1 degree increments: 0, 1, 2, ... 359 degrees. But there is is nothing magical about that increment. Instead, you might try 5 or 10 degrees. Find the best match, then try angles on both sides of that match.

Or you can use a multiscale approach: when comparing images, first "-resize" them by 25%. This takes 1/16th of the time. Again, this gives an approximate solution, so narrows the search space for a non-resized comparison.

[fmw42]

See viewtopic.php?f=2&t=31938&p=146215#p146215. My command now works in the latest beta so that all commands are inline in the one compare and not external processing is needed to rotate the smaller reference image.

Code: Select all

compare -metric rmse -subimage-search -dissimilarity-threshold 1 \
\( 370669patch.png +repage -background black -alpha background \) \
\( 224977object4.png +repage -background black -alpha background -rotate 180 -trim +repage \) \
diff.png

4808.31 (0.0733701) @ 269,40

[cowboyzed]

fmw42 and snigbo,

thank you very much. Now my script run faster (from 2400s to 500s)



PS : I will install the beta version later. I hope the new command will increase performance.

[fmw42]

The following now works in the latest release of IM 6.9.8.6.

Code: Select all

compare -metric rmse -subimage-search -dissimilarity-threshold 1 \
\( 370669patch.png +repage -background black -alpha background \) \
\( 224977object4.png +repage -background black -alpha background +distort SRT 180 -trim +repage \) \
diff.png

In IM 7.0.5.7, you need to turn alpha off at the end or the diff-1.png image will be transparent.

Code: Select all

im7 magick compare -metric rmse -subimage-search -dissimilarity-threshold 1 \
\( 370669patch.png +repage -background black -alpha background \) \
\( 224977object4.png +repage -background black -alpha background +distort SRT 180 -trim +repage \) \
-alpha off diff.png

[cowboyzed]

I have ImageMagick 7.0.5-5 Q16 x86_64.

I want to know : is this last command get the job faster than previous "convert" ?