Scientists uncover invisible motion in video.

[Werty]

NewYorkTimes have an interesting article up about "invisible motion" in videos, dont forget to click the links in the article for more indepth explanation, more videos and also source code (matlab).

http://bits.blogs.nytimes.com/2013/02/2 ... hackernews

They use 4 video frames, wonder what it would take to achive the same with imagemagick.

[snibgo]

I haven't looked at the technical details of this, but the principle seems straightforward. For colour changes, simply measure and extrapolate. The job is harder for motion: identify the movement of individual features, and exagerate that movement.

Ideally, the camera would be stationary. From frame to frame, we can easily identify which pixels don't change. For those that do, we crop a small area from one frame and locate it in the other. Do this for enough areas, and we have a displacement map. Multiply the map by a desired factor and use this to recreate a frame.

If the camera is hand-held, we can use a similar technique to identify the camera motion and cancel it. If the camera is pointing towards a person's face with no detail in the background, and the camera or face is moving, then cancelling the movement is harder.

[anthony]

This sounds very interesting, though generating a full displacement map seems overkill. Often you only need a rough idea of movement.

If you have knowledge of what part is background the movement vector could be determine by the change in the edge of the difference between the two frames and the known static background.

EG: get the difference from two frame and background, look for largest edge displacement.

However if you really wany a full displacement map (and you don't have the background image) you can speed up that map generation by limiting the sub-image searches to areas that actually changed by some large amount.


However what is being talked about is extremely fine movements.

[snibgo]

Yes, many simplifications can be made. Cameras with anti-shake technology sometimes (always?) assume corrections can be made by simply moving the image.

Video technology already knows about movement, because its useful for compression, so that's the obvious an obvious route for implementing this. I wouldn't be surprised if someone has already written an ffmpeg filter.

[snibgo]

I was forgetting that a program I've had for years, called SMPlayer, can show movies superimposed with a grid of about 20x16 motion vectors. Clever stuff.

[anthony]

The job is harder for motion: identify the movement of individual features, and exagerate that movement.

Just remember that displacement maps are reversed. They do not define how something should move, but where something should come from. That is the vectors are for destination, and not source images.

Conversion from a source movement to destination lookup should howver be straight forward.
1/ create a X and Y gradient image (say red for x an green for y)
2/ each pixels in the source image to the destination (there may have overlaps so priority to the stronger change)
3/ fill in any pixel that became a hole using some type of interpolation (example blur to fill transparency, then remove transparency)
4/ convert the new gradient (absolute lookup map) into a displacement map (relative lookup map)

The last is covered in IM Examples. The former (step 2) isn't covered, though really it isn't too difficult, just tricky...
See Distorts, Forward mapping
http://www.imagemagick.org/Usage/distor ... rd_mapping