Find markers

[elegos]

Hello!

I want to create a gallery which asks the user to take a screenshot of a 3rd-party 3d model viewer and then to send it to complete the submission of his/her work. My idea is to take these "raw" screenshots and then check for the markers (25px*25px 90°magenta angles) and then compare the image with the two markers (png/transparent files) trying to find them, so that I've got the X and Y coordinates of the inner rectangle to maintain, removing all the outside content. So far I've thought about getting pixel-per-pixel color via the command convert filename.png txt:- 2>/dev/null | grep magenta to get all the magenta pixels, and then check this data against the same convert operation, but with the two markers: if the coords but an offset match, I've found the two points. But I don't know what's the best approach to do this (BASH or PHP, if possible). Second part (should be relatively easy) is to get the inside rectangle and make a separate image out of it.

Is there a cleverer way to achieve this? Maybe an already implemented algorithm?


Example of a screenshot with the two magenta markers:
https://dl.dropbox.com/u/1512702/tmp/ma ... xample.png

The two markers:
https://dl.dropbox.com/u/1512702/tmp/vi ... p_left.png
https://dl.dropbox.com/u/1512702/tmp/vi ... _right.png

Thanks!

[fmw42]

Will the marker always be the same size? If so, you can threshold on magenta to make a binary image, white for marker and black for everything else. The make a small image of your marker only converted to white on black. Then use the small image and the large image and use compare to find the upper left corner match points. IM will give you the location of only one, but you can use the match score image to find both as those will be the two brightest points. Be sure to add -subimage-search to the command as that is needed in more current versions of IM.
Once you have both match points, you can offset the upper left one by the height of small image to get the bottom right corner.

see
http://www.imagemagick.org/script/compare.php
http://www.imagemagick.org/Usage/compare/
http://www.imagemagick.org/script/comma ... age-search
viewtopic.php?f=1&t=14613&p=51076&hilit ... ric#p51076

See my script, maxima, for use in extracting the two match points. http://www.fmwconcepts.com/imagemagick/maxima/index.php

[elegos]

Yes, the markers will always be of the same size. I tried using compare with -subimage-search option to search for the first marker, but it returned a false positive (somewhere around a menu item corner). I then tried to search for a chess-magenta/dark-magenta marker, it was better but still not perfect (it found the 1st with a ~15 pixels both X and Y offset). I've then tried witha square 25px magenta marker and IM returned the two images are too different for comparison. I'll try your method. May you please explain the procedure better? I know how to read a manual but I'm little bit confused about which image do what

Thanks

EDIT: I found your script "locatecolor" which returns the first marker. I can create markers of different colors, so that I know that the last coord of the top/left result is where the rectangle starts, and the first of the bottom/right is the end of the rectangle. I'm only afraid about false-positives.

[fmw42]

add -dissimilarity-threshold 100%

see
http://www.imagemagick.org/script/comma ... -threshold

If you post a link to your input image and your small template and your command lines, then I can see if I can help.

Using two different colors may work, so that you can do two compares, to get the different corners.

The main issue is that you may have magenta in your image, so that could cause false positives, but having a unique shape for the template (smaller image) should mitigate that.

[elegos]

convert test.png -channel R -threshold 100% -channel G -threshold 0% -channel B -threshold 100% test_magenta_mask.png

Test image: https://dl.dropbox.com/u/1512702/tmp/magenta/test.png
Output: https://dl.dropbox.com/u/1512702/tmp/ma ... a_mask.png

Mask to compare: https://dl.dropbox.com/u/1512702/tmp/ma ... r_mask.png

compare -metric RMSE -subimage-search test_magenta_mask.png marker_mask.png result.png

Result (after ~1 minute): compare: ImagesTooDissimilar `test_magenta_mask.png' @ error/compare.c/CompareImageCommand/953.

No result.png image

EDIT: with -dissimilar-threshold 100% it will match the 0,0 point (which is way wrong). Also, I can skip the convert part in this way, as the result is always the same (0,0).

[anthony]

You may like to try using -mogrify Correlate With just the binary shape to look for...
http://www.imagemagick.org/Usage/convolve/#correlate

That should clean up the image enormously. and generate strong magenta pixels at the marker locations.

[fmw42]

convert test.png -channel R -threshold 100% -channel G -threshold 0% -channel B -threshold 100% test_magenta_mask.png

Test image: https://dl.dropbox.com/u/1512702/tmp/magenta/test.png
Output: https://dl.dropbox.com/u/1512702/tmp/ma ... a_mask.png

Mask to compare: https://dl.dropbox.com/u/1512702/tmp/ma ... r_mask.png

compare -metric RMSE -subimage-search test_magenta_mask.png marker_mask.png result.png

Result (after ~1 minute): compare: ImagesTooDissimilar `test_magenta_mask.png' @ error/compare.c/CompareImageCommand/953.

No result.png image

EDIT: with -dissimilar-threshold 100% it will match the 0,0 point (which is way wrong). Also, I can skip the convert part in this way, as the result is always the same (0,0).

Your second image should only be the magenta square. But that will not work well because of the threshold to a square will be black and will match lots of black areas.

The best thing would be in this case, assuming you have no magenta anywhere else in large quantities, is to just compare the original image (without alpha channel as I don't know how compare handles that) with the magenta square itself. see my examples at viewtopic.php?f=1&t=14613&p=51076&hilit ... ric#p51076, but make a small magenta square for the second image to replace the small eye and replace your full image for the mandril. Then just add -subimage-search to the command I gave. That is going to find only the first match. So you can either use the match_score (second output image) to find both matches via my maxima script. Or alternately, change your markers to one magenta and another some other color. Then do two search, one with each color square for the second image.

if you either tell me the size of the magenta square or provide that image, I can test with your image, if you still have problems.

[elegos]

anthony: I tried to do

convert -morphology Correlate test.png test_correlated.png

but it will say I'm missing a morphology argument (maybe a kernel image? Don't know what this particular image is).

fmw42: how could I delete the alpha channel in GIMP? You're saying it's not finding the square marker just because it has the alpha channel in it? I could directly make a bmp marker (which shouldn't have alpha), in the end it's just a pixel-per-pixel comparison, isn't it? As of your example, I already saw it and it's just the command I used to try to find the square markers, but as I already wrote, with dissimilar threshold 1 it will find 0,0 and without it won't find it.

The images are linked in my last reply test.png is the image to check, marker_mask.png is the black square (used for comparison after the magenta threshold), while this is the original magenta marker (they're both 25*25 px).

Thanks


P.S.
Eventually I wish there was a fast compare tool not to take around 1 minute per image, as I can't foresee the amount of images my website will have to handle. I wish there was a screenshot tool in Windows like the one on OSX, selecting an area... this would lead people to take screenshots manually without having to edit them (which may be boring and overkill for what I'm doing).

[fmw42]

It works perfectly fine for me.

compare -metric rmse -subimage-search test.png marker.png test_similarity_%d.gif
0 (0) @ 293,182

293,182 is the top left corner of the magenta marker in the test.png image. The match score of 0, means a perfect match. It processed in 44 seconds.

Here is the second similarity image from the output.



Note the two brightest spots are the locations of the top left corner of your marker in the test.png image.

If I use my maxima script:


maxima -r 5 -n 2 test_similarity_1.gif
293,182 gray=65535,255,100%
568,353 gray=65535,255,100%

The 293,182 and 569,353 are the locations of the top left corner of your two markers. The gray= identifies that the score was 100% match (255 is perfect in 8-bits and 65535 is perfect in 16-bits)

If you want speed up, then you could recompile IM in HDRI Q16 mode and use my FFT processing script, normcrosscorr. See

http://www.fmwconcepts.com/imagemagick/ ... /index.php
http://www.fmwconcepts.com/imagemagick/ ... mcrosscorr

[anthony]

Code: Select all

convert -morphology Correlate test.png test_correlated.png

but it will say I'm missing a morphology argument (maybe a kernel image? Don't know what this particular image is).

The command should be something like

convert test.png -morphology Correlate {kernel_with_shape_being_searched_for} test_correlated.png

The 'shape kernel' is a array of values representing a binary image. typically 1 for the search shape and -1 for neighbouring 'background', and 0 for don't care. The example in the link I give shows how it works
http://www.imagemagick.org/Usage/convol ... ate_search

It should be faster than sub-image search. Note that the 'shape' is basically binary with not color reference. As such it will find that shape in ALL colors. The basic color of a shape will be represented as pixels in the resulting correlation image.

A simplification is to first generate a image with white for magenta (desired color), and lower values for picels with colors less 'magenta-like'. As it is greyscale, you can then restrict the search to one grayscale channel and get a 3 times speed up.



For larger 'shapes' (say 50 pixels or more wide/high) you can actually do correlation using a Fast Forier transform (though I don't have an example of this). To use FFT well (especially with negatives for background matching) it is a good idea to use a HDRI version of ImageMagick.

Using FFT also goes further in allowing you to search for shapes of any scale or even rotation.

Fred should certainly be able to generate a example of this. It is something that I should add to IM Examples, but have has no time to 'play' as such

[fmw42]

Using FFT also goes further in allowing you to search for shapes of any scale or even rotation.

Fred should certainly be able to generate a example of this. It is something that I should add to IM Examples, but have has no time to 'play' as such

Current implementations of FFT are not scale and rotation invariant. That would require some other transform, which has not been implemented.

My script normcrosscorr, does what Anthony is discussing with the current FFT processing. It does a correlation in the FFT domain and tries to do it normalized for mean and std so values are in range 0 to 1.