Using IM to subtract changes in photos

[andrewgroup]

Having just discovered IM and reading it's capabilities I quickly realize it may do exactly what I need, but haven't a clue how to accomplish what I want. Let me describe the goal.

In the world of Video security you have 2 common Video Encoding methods.
Mpeg2, Mpeg4

I want to create a sample series of images to simulate the idea of Mpeg2,4 and H.264 by taking a 3 second exposure at 15 frames (real 4MP JPEG photos. These 45 images will be an image of someone running across the frame, and the entire background will be static.

MP2/MP4/H .264 encoding creates specific frame type
I frame is a full image encoded and saved an can be recreated on it's own The first image will be an I-Frame
In my example the next image will be a full JPEG, but I would like to be able to remove from the second image everything that did not change and leave only the pixels that changed.. this would be my sample B-Frame..

now that I have my first I frame and B frame, I would like to create a sample P or Predicted frame...

Subtract what did not change from the third image what did not change from the first image, and the second image..

continue doing this for the 13 example P frames. Print image #16 as the first I-frame, for the beginning of the next Second, and repeat the process...

While I know these are not true Frames, based on the encoding used by MP2,MP4, or H.264, but being able to put printed images into the hands of people will give a very visual example of how the encoding process works.

Any thoughts?

[fmw42]

I really don't know much about video. But if you want to compare two images each with one frame/layer/page only (that is convert your video into individual images) you can do that with IM compare function, if the two images are the same size.

compare -metric rmse image1 image2 diffimage

see
http://www.imagemagick.org/script/compare.php
http://www.imagemagick.org/Usage/compare/

The various metrics or statistics can be found in the second reference.

An alternate is to just use -compose difference (or with IM compiled in HDRI so the negative values are saved, use -compose minus).

see
http://www.imagemagick.org/Usage/compare/#difference
http://www.imagemagick.org/Usage/compose/#compose
http://www.imagemagick.org/Usage/compose/#minus
http://www.imagemagick.org/Usage/compose/#difference


You may need to work in HDRI mode, so that you can keep the negative as well as positive difference, if you need to add them back later to another image.

P.S. I have not tried this, but if you convert your video to say TIFF format so that each frame of the video is a layer of the one tiff image, then you may be able to get the comparison per layer or difference per layer or minus per layer if you save the result as tif as well.

[andrewgroup]

I am trying to create a visual representation of what happens MP2/4 H.264 Video Encoding.
I have a high speed camera capable of 15 to 30 images per second and each would be a full frame Jpeg image... I want to use IM to simulate the video compressing process to illustrate that a Video Surveillance system claiming 30 frames per second, while this sound impressive, it may actually only be 1 true self standing frame per second. The 29 other frames taken in that second are not actually frames thats can be extracted from the video. This type of video compression is getting challenged in the courts, since the majority of frames are actually mathematical equations unsing an interger based Cosine transform function.. blah, blah, blah...

So taking these 45 images of a scene and using IM, I think I can simulate how these compression methods work..

I'll give this a try, and if I find myself in over my head, perhaps I can compensate someone more experienced wit IM to help..

Thanks

[snibgo]

It occurred to me that ffmpeg might have the ability to extract "raw" frames, that would visually show the data contained in I, B and P-frames. However, a quick look at the help ("ffmpeg -h") doesn't mean much to me.

As fmw42 says, IM can easily identify what has changed between frames, such as by using "-compose Difference". That gives me a simple mask that shows where the image has changed. When I've done this, using frames extracted by ffmpeg from movies, I need to remove noise first. For your purposes, such niceties probably don't matter.

I don't know enough about the internals of MPEG files to know what B and P-frames should look like. If you can show us, and provide sample "with person" and "without person" frames, doubtless we could knock something up.

[fmw42]

Sorry I know little about MPEG format. Perhaps someone else on the forum may be able to help more.