Extract text from an image

[kereilly]

So I want to extract the text from an image. Seems doable since i can get multiple images that are different, except for the text that i need. Sample:
I think I should be able to compare the two images and discard what is different, leaving the text. I did a forum search and this was pretty much answered here: viewtopic.php?f=1&t=15584&start=15#p55396
In this case they wanted what was different rather than the same.The summery from user fmw42:

convert image2.png image1.png -alpha off +repage \ <--- read images and turn off any existing alpha channel and remove virtual canvas using +repage so that the image sizes are the true canvas sizes
\( -clone 0 -clone 1 -compose difference -composite -threshold 0 \) \ <--- copy the two input images, get the absolute difference image and threshold to black/white so that white is any difference and black is no change. This becomes a mask.
\( -clone 0 -clone 2 -compose multiply -composite \) \ <--- multiply the mask against a copy of image2 so that the difference areas remain and the rest is turned black, which is the color I deduced was not a current color in the image any where.
-delete 0,1 +swap -alpha off -compose copy_opacity -composite -trim +repage \ <--- delete the original two images, turn alpha off, swap the order as needed for the compose, then put the mask as the alpha (trasparency) channel of the image with the changes surrounded by black, trim and reset the virtual canvas (typical when trimming)
image12diff2a.png <--- write output image

To get what is the same in each image i should just be able to invert the mask and this command should work. However I don't get a good mask it looks like this:
After some research it seams difference is very exact and any mathematical difference in color is seen. These images come from compressed mpg video so i'm sure the compression artifacts is whats causing the discrepancy. I tried adding fuzz to counter this but no real difference.
\( -clone 0 -clone 1 -compose difference -fuzz 10 -composite -threshold 0 \)
Am i using fuzz wrong? Or should i be going about this a different way. Also i can extract more than two images from the video. Can difference take more than two sources for more datapoints? Here are the two original images:


Any help is apprecieated - Drop box url's don't appear to work. I'll try to fix that

Version: ImageMagick 6.9.0-0 Q16 x86_64 2015-07-27
url to images:
https://www.dropbox.com/s/jfo4guecqn175 ... e.png?dl=0
https://www.dropbox.com/s/z89vi8mcvmayo ... 1.jpg?dl=0
https://www.dropbox.com/s/ubrmoiohz9nt4 ... 2.jpg?dl=0
https://www.dropbox.com/s/s607p2a2g1zze ... 2.gif?dl=0

[fmw42]

Some of your links are broken, so we cannot see the issues. Also your first link at the bottom shows a mostly empty white image with some black at the borders. What is that for?

[kereilly]

I don't know why these links are broken for the img tag. They work for me when i make them a link at the bottom.
The first link is the mask that gets created
The second link is the first frame of video
the third link is the second frame of video
the fourth link is a gif of the two to show that the text in the image is in the exact same spot and same color (at least to the naked eye)

The mostly white image (first link) gets created when i run
"convert image2.png image1.png -alpha off +repage \
\( -clone 0 -clone 1 -compose difference -composite -threshold 0 \) difference.png"
The expected result is the text that is in image1 and image2 would be black in difference.png since those are the only pixels that are the same. if I understand your instructions white is different and black is no change. Ideally i'll get only the text against a white background so i can send them to a command line OCR software. Which link at the bottom didn't work for you?

[snibgo]

Looking at image1.jpg and image2.jpg, magnified to 400%, the colours in the "white" text are very different. Many areas of the images have a similar difference.

Instead of single frames, you might take averages from two sets of frames, ensuring that no two frames in a set are similar overall (eg two frames of the same newscaster would be similar overall to each other). Then find the difference:

Code: Select all

convert in1.png in2.png -compose Difference -composite -threshold X% out.png

Don't extract mpg frames to jpg unless you are certain it is lossless.

[kereilly]

Okay now i'm making progress. I didn't realize i use -threshold to change the tolerance. Also i extracted my video files to png rather than jpg. Reading ffmpeg howto's this appears to save the image lossless. I'm a little confused at your other suggestion:

Instead of single frames, you might take averages from two sets of frames, ensuring that no two frames in a set are similar overall (eg two frames of the same newscaster would be similar overall to each other). Then find the difference:

So i should extract four frames from the video, compare the first two, then the second two and then compare the difference from both sets? I tried this and it didn't produce desired results:

Code: Select all

convert one.png two.png -compose Difference -composite -threshold 15% diff1.png
convert three.png four.png -compose Difference -composite -threshold 15% diff2.png
convert diff1.png dif2.png -compose Difference -composite -threshold 15% out.png

Since the difference comes out as black and white i can't really compare diff1 and diff2. Should i be making a mask over the original and compare two sets that way?
https://www.dropbox.com/s/qn7fkbp7b5dh0zh/one.png?dl=0
https://www.dropbox.com/s/p1bg2d08vs0xhcd/two.png?dl=0
https://www.dropbox.com/s/kazezb3uu0sdy ... e.png?dl=0
https://www.dropbox.com/s/wdh1i2pxgbnm68q/four.png?dl=0
https://www.dropbox.com/s/5z86qh4lodw9z ... 1.png?dl=0
https://www.dropbox.com/s/q6qbfrxlisue4 ... 2.png?dl=0

[snibgo]

That would be difference of differences, which I'm not suggesting. I suggested difference of averages:

Code: Select all

convert one.png two.png -evaluate-sequence Mean a.png

convert three.png four.png -evaluate-sequence Mean b.png

convert a.png b.png -compose Difference -composite out.png

You probably need many more images than just two in each set. But you have to experiment to find what works for your images. For these four, this is quite good:

Code: Select all

convert one.png two.png three.png four.png -evaluate-sequence Min b2.png

[snibgo]

Did you try my "Min" command? Follow that with a threshold or level, and negate if you want black on white, eg

Code: Select all

convert one.png two.png three.png four.png -evaluate-sequence Min -threshold 60% -negate b2.png

What more do you want?

[kereilly]

Okay your last line does the trick. I did put in "Min" before but that was it. I thought it was part of -evaluate-sequence and didn't know i had to add threshold to it. With the negate i get excatly what i was looking for. Thank you very much i'll just experiment to find the right amount of frames and threshold levels from here. Appreciate the time you took to help me.