Legacy ImageMagick Discussions Archive

Howdy all!
First post in the IM forum here. If I'm in the wrong subforum, please let me know. Everyone hates being the clueless new guy.

I'm a recreation ecologist with a software background and I'm seeking to augment some machine vision work in my research group, using ImageMagick. The big picture view, no pun intended, is to use cellphones and other off the shelf imaging hardware to do structure-from-motion based 3D outdoor scene reconstruction and analysis. At this point, all the analyses are post-hoc, so there's no practical constraint on storage space, processor power, or batch processing length of time. I.e., no realtime optimization needed. We're using OpenCV for some parts of the process already.

Task: perform basic image content matching against a human-tagged large set of reference images.
Pipeline:

• tag large sets of reference images for key attributes
• auto-upload files to local OSX Apache2 webserver (done),
• organize image files into directories using PHP (done),
• extract basic summary stats for each image using Imagick and PHP exec(ImageMagick via command line, OpenCV via command line) (done),
• compare each new image against tagged images to identify (working on it)
  • presence/absence of color families (e.g., bark browns vs. soils, lichen greens vs. leaf greens, etc.
  • presence/absence of human figures, generally viewed in profile/side-on angle
• generate report and display to user through email and/or web interface (done).

So, what is the wisest way to go about using ImageMagick to guess about new images and propose defined tags for them relative to a reference set of tagged images?

The fundamental question is: identifying what is the camera/remote-operated drone/unmanned vehicle looking at?

Our primary difficulty is irregular lighting. Many of our images are in woodland (i.e., full shade -> partial sun/hard shadow -> full sun with no way to predict. I suppose that could be another class of pre-trained images... We have only limited control over each photo/video frame's direction & orientation, so we're trying to grapple with shape recognition as well.

I'm familiar with Fred's excellent collection of scripts. Should I be doing histogram comparisons? the Similar script? Something else?

Thank you and I am happy to provide clarifications if I'm not explaining this well.

This is a problem that has been studied quite a bit using multispectral images and by using color classification techniques, e.g. isoclass, supervised classification, etc. But IM does not have anything like that. There is a color segmentation, but it is rather crude (see -segment). I suspect OpenCV may have more relevant functions, but do not know for sure. With IM, the best I can think of would be to reduce colors in the image and compare textual histograms of the colors. My colorspectrum script might help in combination with the IM compare function?

It might be nice to see examples of the images your are trying to compare. You can upload them to any free hosting service such as dropbox (public folder) and put links here.

Hi, Fred,
Thanks for your speedy reply; big fan of your collection of scripts. It is clear a lot of work and love has gone into them.

Consider the example image set below (sizes are huge, sorry!). All but the penultimate/second-to-last contain a human-created foot trail, and each photo has different camera angle, lighting, autoexposure, etc. Lets say we tag all but that one in our database as "has foot trail". Given the different color of trampled soil in the trail treadway vs surrounding areas, what approach among your scripts or related algorithms can best evaluate the non-trail photo and reject suggesting to tag it like the others, against some specified certainty threshold?

Perhaps another example might be an arbitrarily large collection of photos with people in them: portraits, walking down the sidewalk, sitting in an office, etc. All of these images tagged as "has person in frame" manually by a human. The user feeds an image into a comparison algorithm that looks for matches against the color family present in all the tagged photos: skin tone. There will be millions of other, random colors, but the skintones are identified and used as the accept/reject criterion for suggesting to tag the new image as "has person in frame".

Supplying enough tagged images should not be a problem.

So maybe something like,

• Quantize the histogram for all the "has person in frame" tagged images and the new unknown image
• Compare the new image's histogram against that of each of the tagged images
• Compute some goodness of fit statistic for each pairing and across pairings (RMSE?)
• Do the same against a sequence of histograms from images tagged "does NOT have person in frame"
• Accept or reject tagging the unknown image based on which set of histograms was a closer match

Conceptually, it seems like this might work, but I'm actually just interested in the color families, not really the entire histograms...

I really do not know. There are some references if you search with Google for detecting skin tones. That would be where I would start. OpenCV has some facial recognition, but I am not sure that will do what you want.

I do not think any of my scripts will really help much.

See also
http://www.imagemagick.org/Usage/compare/#type_general
http://www.imagemagick.org/Usage/compare/#metrics