Compare subimage-search

[queensoft]

Tried several things, metrics, a BAT script (don't remember the link right now), image crop, different similarity thresholds (some very low)... nothing helped.
The best result I got is a complete mismatch - it says found match at 0,4, which is obiously wrong.
The other results are taking too long or "images too dissimilar".
I kinda need a processing time under 1-2 minutes, if possible.

Sub-image to search for - is is extracted from the original JPG below, clean noise jpg artifacts.
I can get a much cleaner original PNG.


Original image - yes, it's huge:
http://i61.tinypic.com/izqrko.jpg

Crop 1 - from top left (0,0) and adding around 100 pixels extra space:
http://i59.tinypic.com/16kchte.png

Crop 2 - 100 pixels extra space around the target area:
http://i62.tinypic.com/2dkjml1.png

Command line used (or some slight variations):

Code: Select all

compare -subimage-search -metric ae -similarity-threshold 0.8 testing1_Page_22.png first_name_02.png null.jpg

I use ImageMagick-6.9.0-Q16, Windows 7 x64.

[snibgo]

First: AE is not a good metric for this, as a slight shift in colour counts as bad as a change from black to white.

Second: "-similarity-threshold 0 -dissimilarity-threshold 1" would be useful to you.

Third: your images are at different scales. IM's compare won't, by itself, adjust the scale of one image to search in another. It simply compares pixels, as they are.

Fourth: your images are large. When the correct scale is known, I would resize each by the same amount so the minimum dimension is 10 pixels. The search is then much faster, but the result is approximate. Crop the large image to suit, then search again without resizing.

I give scripts for this on my site, including variable-scale searches. They are Windows BAT syntax.

[queensoft]

Metric - tried all.

"-similarity-threshold 0 -dissimilarity-threshold 1" - I will try this.
So far, tried -similarity-threshold from 0.2 to 0.99

Images are at different scales - no, they are not.
Small image is exracted from large image.
No resize was done on either image after extraction.

Images are large, resize to minimum dimension is 10 pixels -
The small image (to be searched), is now 210x84 pixels.
Resize would make it 25x10 pixels and completely destroy the text.
Already tried crop, 2 different crops, images are attached, but results are also totally wrong.

Scripts - right, already tried yout BAT file: http://im.snibgo.com/srchimg.htm
But, sorry to say this, not working at all.
I mean, no bad results, I mean BAT not working.
First of all: @call echoOffSave - I do not have that file/function.
I have foud your script directly from google search, not your main page (with other instructions).
Maybe on the main page it says : "you need to also download this and that....".
Full output from script:

Code: Select all

test02 testing1_Page_22.png first_name_02.png

Code: Select all

rem  Searches image testing1_Page_22.png for image
first_name_02.png.
'echoOffSave' is not recognized as an internal or external command, operable program or batch file.

call setInOut testing1_Page_22.png si
'setInOut' is not recognized as an internal or external command, operable program or batch file.

if "" == "1" (
set DEBUG_OUT=_si_dbg
 if "" == "" set siDEBUG_STROKE=-fill None -stroke #f00
)

set SRCNAME=testing1_Page_22

set SRC=

set SUB=first_name_02.png

set fBAT=

if "" == "" set fBAT=NUL

if "" == "" set siMETRIC=RMSE

set TMPEXT=.miff

set TMPDIR=d:\_Temp\

set TMPSRC=d:\_Temp\\siSrc.miff

set TMPSUB=d:\_Temp\\siSub.miff

if not "" == "0" (
del d:\_Temp\\siSrc.miff  2>nul
 del d:\_Temp\\siSub.miff  2>nul
)

if not "NUL" == "NUL" del NUL  2>nul

FOR /F "usebackq" %L IN (`convert -ping first_name_
02.png -format "subW=%w\nsubH=%h\nMinDim=%[fx:w<h?w:h]" info:`) DO set /A %L

set /A subW=211

set /A subH=84

set /A MinDim=84

FOR /F "usebackq" %L IN (`identify -format "OFFSET_
X=%X\nOFFSET_Y=%Y" `) DO set /A %L

echo MinDim=84 Offset=(,)
MinDim=84 Offset=(,)

if 84 GEQ 10000 (
call :ResizeSrch 0.1 1000
 call :CropSrch !COMP_XW! !COMP_YW! 2000
)  else if 84 GEQ 5000 (
call :ResizeSrch 0.2 500
 call :CropSrch !COMP_XW! !COMP_YW! 1000
)  else if 84 GEQ 2000 (
call :ResizeSrch 0.5 200
 rem call :CropSrch !COMP_XW! !COMP_YW! 400
 call :CropResizeSrch !COMP_XW! !COMP_YW! 100 5 20
 call :CropResizeSrch !COMP_XW! !COMP_YW! 40 10 10
 call :CropResizeSrch !COMP_XW! !COMP_YW! 20 20 5
 call :CropResizeSrch !COMP_XW! !COMP_YW! 10 50 2
 call :CropSrch !COMP_XW! !COMP_YW! 4
)  else if 84 GEQ 1000 (
call :ResizeSrch 1 100
 rem call :CropSrch !COMP_XW! !COMP_YW! 200
 call :CropResizeSrch !COMP_XW! !COMP_YW! 100 5 20
 call :CropResizeSrch !COMP_XW! !COMP_YW! 40 10 10
 call :CropResizeSrch !COMP_XW! !COMP_YW! 20 20 5
 call :CropResizeSrch !COMP_XW! !COMP_YW! 10 50 2
 call :CropSrch !COMP_XW! !COMP_YW! 4
)  else if 84 GEQ 500 (
call :ResizeSrch 2 50
 call :CropResizeSrch !COMP_XW! !COMP_YW! 100 5 20
 call :CropResizeSrch !COMP_XW! !COMP_YW! 40 10 10
 call :CropResizeSrch !COMP_XW! !COMP_YW! 20 20 5
 call :CropResizeSrch !COMP_XW! !COMP_YW! 10 50 2
 call :CropSrch !COMP_XW! !COMP_YW! 4
)  else if 84 GEQ 200 (
call :ResizeSrch 5 20
 call :CropResizeSrch !COMP_XW! !COMP_YW! 40 10 10
 call :CropResizeSrch !COMP_XW! !COMP_YW! 20 20 5
 call :CropResizeSrch !COMP_XW! !COMP_YW! 10 50 2
 call :CropSrch !COMP_XW! !COMP_YW! 4
)  else if 84 GEQ 100 (
call :ResizeSrch 10 10
 call :CropSrch !COMP_XW! !COMP_YW! 20
)  else if 84 GEQ 50 (
call :ResizeSrch 20 5
 call :CropSrch !COMP_XW! !COMP_YW! 10
)  else if 84 GEQ 20 (
call :ResizeSrch 50 2
 call :CropSrch !COMP_XW! !COMP_YW! 2
)  else if 84 GEQ 10 (
call :ResizeSrch 50 2
 call :CropSrch !COMP_XW! !COMP_YW! 2
)  else (FOR /F "usebackq tokens=1-4 delims=()@, " %R IN (`compare   -similarity
-threshold 0 -dissimilarity-threshold 1 -subimage-search   -metric RMSE first_na
me_02.png NULL: 2>&1`) DO (
set COMP_INT=%R
 set COMP_FLT=%S
 set COMP_XW=%T
 set COMP_YW=%U
) )

rem 20 is percentage reduction, eg 50 for 50. May b
e <1.

rem 5 is multiplier (=100/20)

echo ResizeSrch 20 5
ResizeSrch 20 5

rem The convert and compare could be combined into
one convert.

set COMP_XW=0

set COMP_YW=0

set TMPSRC_RES=d:\_Temp\\testing1_Page_22_si_20.mif
f

if not "" == "0" (del d:\_Temp\\testing1_Page_22_si
_20.miff  2>nul )

if not exist d:\_Temp\\testing1_Page_22_si_20.miff
convert  +repage -resize 20% d:\_Temp\\testing1_Page_22_si_20.miff
convert.exe: no images defined `d:\_Temp\\testing1_Page_22_si_20.miff' @ error/c
onvert.c/ConvertImageCommand/3210.

convert first_name_02.png -resize 20% d:\_Temp\\siS
ub.miff

set COMP_XW=

FOR /F "tokens=1-4 usebackq delims=()@, " %R IN (`c
ompare   -similarity-threshold 0 -dissimilarity-threshold 1 -subimage-search   -
metric RMSE d:\_Temp\\testing1_Page_22_si_20.miff d:\_Temp\\siSub.miff NULL: 2>&
1`) DO (
rem echo %R %S %T %U
 set COMP_INT=%R
 set COMP_FLT=%S
 if not "%T" == "" set /A COMP_XW=%T*5
 if not "%U" == "" set /A COMP_YW=%U*5
)

(
rem echo compare.exe: unable to open
 set COMP_INT=compare.exe:
 set COMP_FLT=unable
 if not "to" == "" set /A COMP_XW=to*5
 if not "open" == "" set /A COMP_YW=open*5
)

if "0" == "" exit /B 1

if not "" == "0" (del d:\_Temp\\testing1_Page_22_si
_20.miff  2>nul )

rem echo   COMP_XW, YW = 0 0

exit /B 0

rem !COMP_XW!,!COMP_YW! are coords for top-left of
estimated result.

rem 10 is plus or minus for tolerance in both direc
tions.

echo CropSrch !COMP_XW! !COMP_YW! 10
CropSrch !COMP_XW! !COMP_YW! 10

set /A cropL=!COMP_XW!-10
Missing operator.

set /A cropT=!COMP_YW!-10
Missing operator.

if 1 LSS 0 set cropL=0

if 1 LSS 0 set cropT=0

set /A cropW=211+10+10

set /A cropH=84+10+10

set COMP_XW=1

set COMP_YW=1

convert      +repage   -crop 231x104+1+1   +repage
  d:\_Temp\\siSrc.miff
convert.exe: no images defined `d:\_Temp\\siSrc.miff' @ error/convert.c/ConvertI
mageCommand/3210.

FOR /F "tokens=1-4 usebackq delims=()@, " %R IN (`c
ompare   -similarity-threshold 0 -dissimilarity-threshold 1 -subimage-search   -
metric RMSE d:\_Temp\\siSrc.miff first_name_02.png NULL: 2>&1`) DO (
rem echo %R %S %T %U
 set COMP_INT=%R
 set COMP_FLT=%S
 if not "%T" == "" set /A COMP_XW+=%T
 if not "%U" == "" set /A COMP_YW+=%U
)

(
rem echo compare.exe: unable to open
 set COMP_INT=compare.exe:
 set COMP_FLT=unable
 if not "to" == "" set /A COMP_XW+=to
 if not "open" == "" set /A COMP_YW+=open
)

rem echo   COMP_XW, YW = 1 1

exit /B 0

rem set /A COMP_XW+=

rem set /A COMP_YW+=

if "" == "1" (
set /A endX=1+211-1
 set /A endY=1+84-1
 rem FIXME: does draw respect offsets? Add +repage to make this irrelevant.
 convert          +repage          -draw "rectangle 1,1 !endX!,!endY!"
)

if not "NUL" == "NUL" echo set COMP_XW=1&set COMP_Y
W=1&set COMP_FLT=unable&set COMP_CROP=211x84+1+1  1>NUL

FOR /F "usebackq" %L IN (`convert -ping first_name_
02.png -format "centX=%[fx:int(1+w/2)]\ncentY=%[fx:int(1+h/2)]" info:`) DO set /
A %L

set /A centX=106

set /A centY=43

if not "" == "0" (
del d:\_Temp\\siSrc.miff  2>nul
 del d:\_Temp\\siSub.miff  2>nul
)

[snibgo]

I can't see how to download the full-size version of the form. When I resize the downloaded version by 157%, I can do the search.

My scripts use each other. You can download a zip of them all from the page http://im.snibgo.com/zipbats.htm . They are assumed to be in a directory called %PICTBAT%. If %PICTBAT% is defined, it must be the directory of the BAT files. Likewise if %IM% is defined it must be the directory containing convert.exe.

The following takes about 4 seconds:

Code: Select all

set SRC=izqrko.jpg
set SUB=kal46t.png

%IM%identify %SRC%
%IM%identify %SUB%

%IM%convert %SRC% -resize 157%% r.png
%IM%identify r.png

call %PICTBAT%srchImg r.png %SUB%
echo %siCOMP_XW% %siCOMP_YW%

It returns the coordinates "139 1197", which is correct for the second occurrence of "First Name". You would need to do further work to find both occurrences of "First Name".

[queensoft]

I will try and let you know.

[queensoft]

Managed to run the script.
But got result: 20 3
Top left corner, which is pretty much what I got using the standard imagemagick command.

[snibgo]

My script usually gives the same result as "compare ... -subimage-search" (but much faster). If it is, but the result isn't what you expect, perhaps you are not using the images you think you are.

[queensoft]

Try it for yourself: https://www.dropbox.com/s/1tunak0ilqfvm ... s.ZIP?dl=0

[snibgo]

Using your files (images and bat files), I get the result "141 471", which is correct for the first occurrence of "first name".

This is with the pre-built Q16 distribution of IM 6.9.0-0, on Windows 8.1.

What do you get from this?

Code: Select all

compare -metric RMSE %SRC% %SUB% -similarity-threshold 0 -dissimilarity-threshold 1 -subimage-search NULL:

It takes 5 min 20 sec, and returns:

Code: Select all

2481.48 (0.0378649) @ 141,471

And what does "%IM%convert -version" give you? I get:

Code: Select all

Version: ImageMagick 6.9.0-0 Q16 x64 2014-11-14 http://www.imagemagick.org
Copyright: Copyright (C) 1999-2014 ImageMagick Studio LLC
Features: DPC OpenMP
Delegates (built-in): bzlib cairo freetype jbig jng jp2 jpeg lcms lqr pangocairo
 png ps rsvg tiff webp xml zlib

[queensoft]

Yes, after about 5 minutes I get the same result: 141,471

I have newer version:

Code: Select all

Version: ImageMagick 6.9.0-3 Q16 x64 2014-12-30 http://www.imagemagick.org
Copyright: Copyright (C) 1999-2015 ImageMagick Studio LLC
Features: DPC Modules OpenMP
Delegates (built-in): bzlib cairo freetype jbig jng jp2 jpeg lcms lqr openexr pangocairo png ps rsvg tiff webp xml zlib

I can use crop to speed up the search.

[snibgo]

Okay, then it seems my script works (for me) for 6.9.0-0 but not (for you) on 6.9.0-3. Sorry, but I have no immediate plans to try out a new version of IM.

[queensoft]

No worries, looks like processing time is the same.
Thanks for pointing me in the right direction.