Which image metrics to use for selecting representative images from large image sets?

[konstantin]

For example when selecting nice images for preview image montages of videos, movies (thumbnails, contact sheets, etc):



Currently I extract about 1500 images equidistantly from a 1.5 hour video, and I calculate the following metrics for every images.

entropy:

Code: Select all

identify -format "%[entropy]" 000001.jpg

number of unique colours:

Code: Select all

identify -format "%k" 000001.jpg

standard deviation:

Code: Select all

convert 000001.jpg -format "%[fx:standard_deviation]" info:

canny edge detector metric:

Code: Select all

convert 000001.jpg -colorspace gray -canny 0x1+15%+40% -format "%[fx:mean]" info:

and last but not least the jpeg file size in bytes.

Then I normalize all of those data series to the 0.0-1.0 interval, and finally I calculate their root mean square (RMS). I normalize the RMS values as well into the 0.0-1.0 interval. Then I simply sort the images in reverse order according to the calculated RMS value, and I select the needed number of images by hand from the first part of the list...

What other image metrics and selecting methods should I consider to automate this process even further, possibly fully automatic?

[snibgo]

Interesting.

After taking your 1500 samples for a 90 minute video (say, one every 3. seconds) you might discard frames that are similar to each other and close together in time.

It is possible to divide a movie into scenes and shots. Shots are easy: adjacent frames are similar. A number of shots make a scene, where the end of one shot is a close match to the start of another, with one or two shots inbetween. This might be used to isolate the most interesting shots in a scene.

You might find faces, and give a high rating to frames with close-up faces. (I don't know the most appropriate software for this.)