Compare (subimage-search) rotated image without intermediary files

[cowboyzed]

Hi all, (excuse my bad english)

With PHP, I have to find if a small picture is repeated in a big picture, whether it's randomly rotated or cropped.

I am looking for an elegant way (faster) to do this.
I want to use command line but I hang myself.

Is it possible to use directly compare -subimage-search with the result of convert -rotate ? (Without using an intermediate file) ?

Code: Select all

shell_exec("compare -metric MSE -subimage-search -dissimilarity-threshold 1 ".$big_image." (convert ".$small_image." -rotate ".$angle360_step_by_10." ) null: 2>&1");

Code: Select all

$objects = slice_image_to_objects("object", $big_image, 64, 64);
foreach ($objects as $object) {
	foreach ($angles as $angle360_step_by_10) {
	        shell_exec();
	}
}

Of course, my code blocks at : (convert $small_image -rotate $angle)

[snibgo]

Is it possible to use directly compare -subimage-search with the result of convert -rotate ? (Without using an intermediate file) ?

If shell_exec permits back-tick embedded commands, you could do it that way. You can "-compare" within "convert" or "magick", but this doesn't return the offset.

But that wouldn't be fast or elegant. You would need one compare per angle. If the subimage is small, this may be acceptable.

For finding a subimage at any rotation and translation, I think the method shown in my page By FFT, what rotation? will work, but I haven't tried that. The method works fine for photographs, but may not for graphics.

If you can link to a sample image and subimage, I can give it a try.

[cowboyzed]

Here is the sample image : (subimage = 64px. the program should return the coordinates of the second heart, or just tell it is the 6th object)


I think shell_exec can execute all commands just like it's done via Terminal. So it permits back-tick embedded commands ! Please can you help me to find the right command to use with compare and convert. I'm sure this is about redirection and pipes, but I didn't find the right place for convert -rotate.

compare -metric MSE -subimage-search -dissimilarity-threshold 1 BIG_IMAGE.PNG (convert OBJECT.PNG -rotate 10) null: 2>&1

I think your solution By FFT, what rotation? is great, but too sophisticated for this case. Since I must transform the BAT file into linux bash

[fmw42]

Where is the object.png example? Please provide that image for testing.

[cowboyzed]

I split this BIG_IMAGE.PNG into 9 parts to obtain OBJECT_1.PNG ... OBJECT_9.PNG. So the program aims to detect that the OBJECT_6.PNG is a copy (rotated) of OBJECT_3.PNG

[snibgo]

If you search for a small image in a larger one, it will find the closest match. If that is an exact match, that will be returned.

I don't understand what you want.

Perhaps you have nine same-sized images, and you want to detect whether any of them are a rotation of another. That's easier than what I thought you wanted; see my What rotation? page.

[cowboyzed]

I want to give it a try but I am not on Windows . As I said : your solution By FFT, what rotation? is great, but I think it's too sophisticated for this case

[snibgo]

I don't understand what you want.

Please show an image that you want to search, and a subimage that you want to search for.

[fmw42]

If you cut your image into 9 equal parts, then you do not need -subimage-search. You should just do a simple two image compare at different rotations of the reference image against each of the 9 parts. This of course assumes the reference Image is one of your 9 cropped images so there is no offset when compared against the correct part.

In unix, the basic command would be

convert image1 \( image2 -background somecolor -virtual-pixel background -distort SRT X \) -metric rmse -compare -format "%[distortion]" info:

Where X is your rotation angle. You just have to loop over image1 and and various rotations of image2. The only issue is what color do you use to fill your rotated image. But that would be your background color from your large image.

[cowboyzed]

Ok, I try to re-explain what I need : I have to detect the subimage that is repeated, whether it's randomly rotated (or cropped).
The method I found is to slice the big image into objects, then loop over these objects and compare per angle.

This command does the job perfectly :

compare -metric MSE -subimage-search -dissimilarity-threshold 1 big_image.png object_1.png null: 2>&1

But instead of object_1.png I need to insert dynamically (if possible) the result of convert -rotate of object_1.png. Otherwise I have to write it in intermediary file.

Thanks

[cowboyzed]

An example of a cropped subimage

[fmw42]

What do the red circles mean? Are you not cropping each row into individual icons and then testing each icon? If you use my command, it should compare each icon at various rotations with each of the cropped icons. It has built in the ability to include the rotate command without using a second convert. Why do you need the subimage-search? It only slows things down and you have to search the whole big image for each small image. My command would be used if you cropped the big image into equal sized small objects. But it will only work if each cropped image has the icon centered along with a properly centered reference icon that is being rotated.

Compare will not allow much in the way of parenthesis processing inline. So that is why I used convert ... -compare. But as snibgo said, currently it does not work (well?) in a subimage-search, since it will not show the matching coordinate.

If you want a faster version of compare that works on unix, then see my normcrosscorr or rmsecorr scripts at my link below. You will have to rotate the reference image your self outside the script. But that should not be an issue, since the script is 100 times faster than compare. Either way, you need to write a script loop over each rotation you want.

You could also post to the Developers forum, to ask that compare be enhance to allow -rotate or -distort SRT inline in a parenthesis.

Note also that -rotate will make your reference image larger, up to 1.414 times at 45 deg. So it might not fit near the edges of your larger image. You should probably do -distort SRT, which will keep the size of your input image. That would be better so long a you have enough background around each icon in the reference image so that it was not cropped by the rotation to the same input size.

[snibgo]

Okay. You have six icons against a white background. Two of the icons are the same, they match, but are rotations of each other.

I assume the red circles show one of the matching pair.

The obvious algorithm, for each group, is:

1. Chop into 6 images. Trim them all. There are 15 possible pairs.

2. Repeat the following for each pair.

3. If the non-white colours in the two images are very different, they are not rotations, so don't test them further.

4. For angle in 0..359, rotate the second image by that angle, and trim it, and compare (eg RMSE score) this rotation against the first image. Record the best score and the associated angle.

5. One pair has a very low score, near perfect match. The other pairs have higher scores. Report the low score and the angle.

Step 3 is optional, but may increase performance.

The method I show in What rotation? will be much quicker than step 4, because it does two distortions and one comparison, instead of 360 rotations and 360 comparisons. However, it assumes the centre of rotation is in the centre of the two images, and this may not be true. But this may be good enough to say correctly which pair match, and approximately what the angle is.

[cowboyzed]

What do the red circles mean?
I assume the red circles show one of the matching pair.
Right! The red circles show the second subimage to be detected.

But it will only work if each cropped image has the icon centered along with a properly centered reference icon that is being rotated.

This is not a solution since, the second subimage could be cropped.

If you want a faster version of compare that works on unix, then see my normcrosscorr or rmsecorr scripts at my link below. You will have to rotate the reference image your self outside the script. But that should not be an issue, since the script is 100 times faster than compare.

So I must write out an intermediary file of each rotation. Ok, I will give that a try.

You could also post to the Developers forum, to ask that compare be enhance to allow -rotate or -distort SRT inline in a parenthesis.

Not the solution anymore (the second subimage could be cropped. See image above)

The obvious algorithm, for each group, is:

1. Chop into 6 images. Trim them all. There are 15 possible pairs.

2. Repeat the following for each pair.

3. If the non-white colours in the two images are very different, they are not rotations, so don't test them further.

4. For angle in 0..359, rotate the second image by that angle, and trim it, and compare (eg RMSE score) this rotation against the first image. Record the best score and the associated angle.

5. One pair has a very low score, near perfect match. The other pairs have higher scores. Report the low score and the angle.

Step 3 is optional, but may increase performance.

Thank you. I will adapt my algorithm that way. Please could you give the optimized commands for Step 3 and Step 4 ? (edit)

( PS : in my first approach, to increase performance, I started the -subimage-search from the object in the right, and crop (delete that object from the big_image) as far as the object to compare progress. )

[cowboyzed]

I forgot to say that the real image will png with a background transparent. But if that changes anything or creates another issue, I will color white the background before all processes