Speed up “movie barcode” generation w/ IM

[killmoms]

If you’ve never encountered them, “movie barcodes” are neat little pieces of digital art generated from source videos. They look like this:

http://moviebarcode.tumblr.com

I wanted to try something involving making an animation based on this movie barcode idea, so (having found tutorials online describing how to dump frames from a video and ImageMagick to process them to generate a barcode) I decided to hack together a bash script to do it. I say “hack together” because I have no formal training in scripting—I mostly accomplished this by watching some tutorials, and a LOT of Googling/copying and pasting/adapting other people’s script code. In any case, the script I have now works, but it’s pretty slow. This isn’t a problem when you’re making one static movie barcode, but when you want to make a sequence of nearly 2000 of them to play as an animation, taking between 1 - 2 minutes to generate each one is prohibitively costly in terms of time. So I was wondering if anyone had any ideas on how to speed this up.

Right now the basic flow of the script is:

1. Take user input to figure out which movie file (currently only supports MKVs) to generate a barcode from, the width of the final barcode (which determines how many frames to dump), and what mode to use (resize/crop/random, which generate 1 barcode; or animation, which generates a sequence).
2. Dump desired number of frames to temporary directory using ffmpeg
3. Use a loop to either resize or crop all the frames without altering the original files (necessary for the animation idea) and then assemble the barcode.
4. If in animation mode, do the above as many times as the original frame is wide.

The specific loop I’m using to make the barcode (in crop mode, since that’s the one I’m using for the animation mode) looks like this:

Code: Select all

FILES=($(find "${1}" -type f | egrep -i "\.bmp$" | sort))

if [[ ${3} == "crop" && -n ${4} ]]; then
	for f in ${FILES[*]}; do
		convert ${f} -resize ${src_frameheight}x\! -crop 0x1+0+${4} +repage miff:-
	done |\
	montage - -mode Concatenate -tile 1x ${2}.png
…
done

mogrify -rotate -90 -sample ${slicecount}x\! ${2}.png

The parameters being passed into the function that contains this loop are:

1. the temporary directory (where the dumped frames live),
2. the output file name,
3. the desired mode, and
4. if the mode is crop-based, a crop offset value (AKA the position in the source frame I want to pull a row of pixels from)

The other modes, random and resize, are elucidated below as the rest of that parent if statement, but follow a similar style, so I haven’t bothered to include them here.

As you can probably see I’m currently doing this vertically—that was my first attempt at optimization, since I read that ImageMagick works on a scanline basis. So now my script outputs the frames rotated 90° clockwise, assembles the barcode vertically, and then rotates it back into the proper orientation at the end. However, this does not seem to have sped things up as far as I can tell. The interesting thing is that neither my CPU nor my I/O appear to be under significant strain during this loop, so I’m really not sure why it’s taking so long. Am I just out of luck in trying to make this go faster using bash scripting? Is this the kind of thing that only a proper, self-contained program in a C-like language using IM libraries could accomplish? Or is there a smarter way to design this loop that I haven’t thought of that would be quicker?

[snibgo]

Interesting. The second image in your link has visible vertical bands, and I suppose these are separate video scenes. Good stuff.

IM algorithms progress in the y-direction, and the x-direction within y. I think storage is likewise. In any case, any savings you get from in-memory proximity of pixels would be swamped by the time for rotations.

What size are the frames? Are they .bmp files? If you have created them, probably quicker to use .miff or .mpc.

.png is slow to write.

How many frames? You could run a single convert command that read a frame, resized/cropped it, repeating for all the frames, then appended the results. This saves calling convert multiple times and piping one miff per frame, but you need to build the command.

[killmoms]

Frames can vary based on the original movie resolution. I’m trying to make the script generic enough that if someone wanted to generate a really huge barcode, they could. So when I output the frames, they’re done at original resolution. Generally that falls somewhere between SD (640 x 480 or thereabouts, depending on aspect ratio) and HD (up to 1920 x 1080). I had been using JPEGs, but figured those were slower to decode, so I switched to BMPs. The problem is, of course, I’m using ffmpeg to generate the frames, so it has no idea what MIFF or MPC are (I don’t think, anyway).

The number of frames can vary based on how wide the user wants the barcode to be. Currently I’m defaulting to 1920 (the ones on that Tumblr I linked are all 1280 wide), but I can specify it at runtime. Conceivably the user could specify a really huge number that would cause almost every frame in the source video to be used, though I haven’t bothered testing anything that large yet.

Can one pass an array of file names into convert? So, I could do something like:

Code: Select all

convert ${ALL_THE_FILES[*]} -${src_frameheight}x\! -crop 0x1+0+${crop_offset} +append barcode.miff

And then subsequently mogrify that miff to rotate, resize, and save as a PNG at the end? Or does this need to be a little more subtle, somehow?

(Now that I think about it, I’m not sure how append works, I’d need to look that up.)

[snibgo]

Convert can't take an array of filenames. (Bash itself might expand the array into the elements; I don't know.) You could do:

Code: Select all

convert @mylist.txt -resize ${src_frameheight}x\! -crop 0x1+0+${crop_offset} +append barcode.miff

where mylist.txt is:

Code: Select all

f_000001.bmp
f_000002.bmp
f_000003.nmp

etc. But this would read all the frames, and then resize and crop each one, appending the result together. Impractical for many frames. (Each pixel takes 8 bytes of memory. So a 2 M pixel frame takes 16 MB. 2000 frames need 32 GB.)

Or you could:

Code: Select all

convert @mylist2.txt +append barcode.miff

where mylist2.txt is:

Code: Select all

f_000001.bmp -resize ${src_frameheight}x\! -crop 0x1+0+${crop_offset}
f_000002.bmp -resize ${src_frameheight}x\! -crop 0x1+0+${crop_offset}
f_000003.nmp -resize ${src_frameheight}x\! -crop 0x1+0+${crop_offset}

This reads a frame, resizes and crops it, keeping the (small) result in memory. Then it reads the next frame, resizes and crops it, etc.

[fmw42]

why not put all the images in a folder and use mogrify to process all the images, except for the append. Then use convert to append them. If you have too many images (too much memory load) to append at one time, you can append them in groups or write a loop to append them one at a time.

[fmw42]

Convert can't take an array of filenames. (Bash itself might expand the array into the elements; I don't know.)

You can do this in Unix, so long as no image has spaces in the name, or you enclose that name with double quotes.

Code: Select all

convert ${arr[*]} -resize 200% result.png

where arr is a list of filenames including the suffixes.

[killmoms]

why not put all the images in a folder and use mogrify to process all the images, except for the append. Then use convert to append them. If you have too many images (too much memory load) to append at one time, you can append them in groups or write a loop to append them one at a time.

Long story short, because the “animation” mode (which was the whole point of me doing this in the first place) relies on those original frame dumps to persist, so that it can iteratively generate many barcodes, one based on each crop offset across the entire image (so barcode 1 will pull from column 0 of every frame, barcode 2 will pull from column 1, etc. across the width of the frame).

Will the append operator in convert simply add the pulled content in that line to the specified output file, and create that output file if it doesn’t yet exist (a la in the first iteration of the loop)?

EDIT: Other idea: append all the original frames together in one giant .mpc and then iteratively pull (either cropping out single rows or resizing given areas) from that to append in sequence? Somehow? Not sure if that would help. I guess it’d eat a fair amount of time making the MPC from the original frames. I’m not aware of any function that’ll pass ffmpeg’s output frames as they stream out of the program through IM to save on that part. Maybe it’s a misguided strategy anyway.

I’ll try the file list idea and see if it’s any faster than what I’m doing now.

[fmw42]

In unix, it seems to me that you could do the following given enough memory to hold all the array files (or use -limit to use disk space if memory is exceeded).

Code: Select all

convert ${ALL_THE_FILES[*]} -resize -${src_frameheight}x\! -crop 0x1+0+${crop_offset} +repage +append -rotate XX -resize WxH result.png

That should take all the files in the array, resize, crop append, rotate and resize and save as png

[killmoms]

Yeah, with 1920+ 6.2MB BMPs I’m thinking that won’t work. I was just hoping there was some way to speed this process along without having to read huge gobs of files into RAM just to extract thin slivers.

snibgo: For some reason when I attempted that text file list idea with the resize/crop commands embedded next to each file, I just got a whole mess of “geometry does not contain image [filename]” errors (I’m guessing from the crop function failing?) and an empty result.png. What have I messed up?

[fmw42]

I just got a whole mess of “geometry does not contain image [filename]” errors (I’m guessing from the crop function failing?) and an empty result.png. What have I messed up?

Try adding +repage after -crop


With so many images, it would seem to me that using mogrify to resize and crop would be the way to go. Then append all the resulting images.

[killmoms]

Did that. In my test script:

Code: Select all

FILES=($(find "${1}" -type f | egrep -i "\.bmp$" | sort))

touch filelist.txt

for f in ${FILES[*]}; do
	echo -e "${f} -resize x504\! -crop 1x0+335+0 +repage" >> filelist.txt
done

convert @filelist.txt +append -sample 320x\! result.miff

The results of that look like this:

Code: Select all

convert: geometry does not contain image `imgs2/mbarcode_000001.bmp' @ warning/transform.c/CropImage/666.
convert: geometry does not contain image `imgs2/mbarcode_000002.bmp' @ warning/transform.c/CropImage/666.
…
convert: geometry does not contain image `imgs2/mbarcode_000321.bmp' @ warning/transform.c/CropImage/666.
convert: geometry does not contain image `imgs2/mbarcode_000322.bmp' @ warning/transform.c/CropImage/666.

I’m guessing it sees mbarcode_000000.bmp fine but then the crop command is messing stuff up afterwards?

The reason I can’t use mogrify is because I need to leave the original frames unchanged so I can process them many times in animation mode, which is the one I need to speed up the most. I COULD use mogrify to resize and crop for the single-barcode modes, potentially, but not for the sequence one. Plus using mogrify would break my “save temp” option, in case the user wants to preserve the dumped frames between jobs or use them for something else.

[fmw42]

for f in ${FILES[*]}; do
echo -e "${f} -resize x504\! -crop 1x0+335+0 +repage" >> filelist.txt
done

convert @filelist.txt +append -sample 320x\! result.miff

This is still going to load all the images with its commands into one convert. IM will try to process previous images with all the commands following it. I do not think this will work.

Try putting all files into a folder and use mogrify to process them to a new folder. Then read all the resulting cropped files in the new folder and append them. Mogrify will only load one image at a time to process rather than loading all the images like convert does.

Alternately, put all the image filename into your filelist.txt (without the commands). Then loop over each line in the filelist and do the convert and save to a different empty folder. Then get all the images in the new folder and append.

[snibgo]

mylist2.txt above, as Fred points out, is wrong. It needs a parenthesis around each line.

Code: Select all

( f_000001.bmp -resize ${src_frameheight}x\! -crop 0x1+0+${crop_offset} )
( f_000002.bmp -resize ${src_frameheight}x\! -crop 0x1+0+${crop_offset} )
( f_000003.nmp -resize ${src_frameheight}x\! -crop 0x1+0+${crop_offset} )

Sorry about that.

[fmw42]

But that will still load all the images in the convert command and you will likely run out of memory, if you have lots of large files.

[killmoms]

Alternately, put all the image filename into your filelist.txt (without the commands). Then loop over each line in the filelist and do the convert and save to a different empty folder. Then get all the images in the new folder and append.

Isn't this basically what I was doing originally, except instead of writing to files I was writing a stream to stdout? Is it just that writing a bunch of files to disk would be faster? Could I use mkfifo to "-append" to a named buffer? Would that be faster or slower than a folder full of files?