mp4 from 1024 mostly clear .PNGs with a static background?

[BrianP007]

I would like to create an MP4 movie with 1024 mostly transparent .PNG files and a static background.

The background would be a full color, up to 36 Mpixel, .JPG. The .PNG files are 1 bit, 0->BLACK 1->CLEAR masks, mostly clear and they are only ~20kB to ~45 kB so they would all fit in memory easily.

One method would be to calculate each bitmask, superimpose it on the .JPG and flatten. That would make 1024 JPGs at roughly 20 MB each would be > 20 GB. If it worked at all, it would take all day to process.

I want to show a large number (at least 256) masks on a regular image and be able to move forward and backward to find interesting combinations.

What is the best ImageMagick way to accomplish this?

[snibgo]

I wouldn't use IM to assemble frames into a movie. ffmpeg is the tool for that job.

You want 1024 frames. Each frame is made from a 36 mpix image, combined under one of 1024 bitmask images (of 36 mpix?) with pixels that are either opaque black or transparent. That seems trivial: just compose each bitmask over the background image, in turn, to make 1024 frames.

I don't suppose your movie is to have 36 mpix frames, so you will be resizing somewhere.

Is that correct, so far?

You are "calculating" the bitmasks. Is that the problem you are trying to solve?

You want to "show a large number" of the bitmasks. You could reduce them so each is one-sixteenth of the screen wide and high, so you have can append them together in a 16x16 array, and view them all at once. Is that what you want?

[BrianP007]

For the movie, I could smush each frame down to 2560x1600, my screen resolution, or perhaps even less. That greatly reduces the data volume.

The bit masks can be anything from brightness level, hue angle (range adjusted from 0-360 to 0-64k), saturation, R, G, B, Luma, Y, U, V, neighborhood contrast or any combination of the above. What does the (red_value, - saturation + 64k ) / 2 look like calculated pixel by pixel? Each would generate a number in the 0-64k range. View it as a grayscale. It is for pure R&D.

My goal is to be able to programmatically separate skin from non-skin, foreground from background, contrasty from bokehd (boken, bokey, bokus?), this color from that color...

The first use will be brightness levels. Bin the data into 1024 bins like the IM histogram does. Black out all 0 bin pixels and see where they are. Then do the 1s, all the way to 1023. Then ranges, say black out bins [0 .. 124]. Now, I have a command line interface and I tell it which bin range to show, but it is somewhat clunky and input intensive.

With so many possibilities, I wanted the movie so I could view the whole thing in 1024/30 = 34 seconds and stop it at an interesting spot and explore more there. I am also going to annotate each frame so I know how it was calculated.

>> append them together in a 16x16 array <<
The ole Matrix trick. That's a good idea! See a Mton of data in a glance...

FFMpeg. I have used it for audio. Checking `ffmpeg -pix_fmts`, it does RGB48, my favorite format, I and O! And it outputs both vdpau_h264 and vdpau_mpeg4 with Hardware Acceleration. Sounds like the right tool.

I will have to build it optimized for my new Skylake/Linux, race it against the venerable, old Sandy_Bridge/win7 binary and hope for a shellacking!

Thanks 1E9!

[snibgo]

In my experience, assembling frames into a movie is fast. The slow part is processing the frames. Various techniques help with that, especially when you have repeated data (eg a fixed background) between frames. The you can write an IM script so it is called just once, and reads the constant data just once. (I don't mean a bash or BAT script, but a file that is called from inside a convert command.)

Small previews help. Do the editing (and choose what effects you want) at a small resolution, and let the computer do the final full-size rendering while you have lunch.

Skip-frames is another editing time-saver. Creating just every other frame is twice as fast, and usually gives all the information you need.

[BrianP007]

>>The slow part is processing the frames<<
My new "hard drive" is completely solid with nothing moving but electrons, the size of a stick of Juicy Fruit, PCIE 3.0 X4 and 2.5 GB/s reads...
- 10 ms read on 72MB, 36MPix, single channel raw (might be already cached).
- 28 ms bin time into 2048 bins [timed with clock_gettime()]
- 62 ms to fill 2048 calloc'ed arrays with the XY coords of each pixel as a uint32, each to the array with the matching brightness level

At this rate, I should be able to do r, g, b, rgb->luma, h, s, v, y, u, v, 24 neighbor contrast, ... pretty quickly. Comparing all 10, 2 at a time is only 45 tests.

>> Skip-frames is another editing time-saver <<
I have a command line parameter -Q for QuantumBits, 8->256 bins, 11->2048. 4 might just be a blur

What I really need is a real-time, 48 BPP, 1500x1000 resolution, Wayland EGL frame buffer with double buffering. Move the mouse and see the result ~instantly without monkeying with file formats, just the RAW! Save only the best masks. Still in the R&D stage

>> Small previews help <<
I build the XY resolution into the level cached extract file header. I think I may use IM->convert to read and downsize the uint48 raw file. It is faster than PerlMagick and spikes all 8 hyperthreads instantly with SMP.

I suspect the IM Gurus are also taking advantage of SIMD, processing at least 1 full pixel/schmear (or more with 5 full 48 bit pixels fitting in just 1, 256 bit, AVX2 XMM register). Using 4 registers for input and 4 for output, 20 pixels could be simultaneously done per batch. 40 with AVX-512!