Blend / Composite Multiple JPEG's into one image

[MitchellK]

Hi folks, I am truly stumped and have now been at this for several hours to try and achieve what I thought would be rather simple.

I have thousands of JPEG's of lightning maps, showing lightning strikes across the country. I simply want to merge them all into image to show all strikes for several months in one image. Right now I have 14,495 images collected over 5 months.

In Photoshop the desired effect I want would be
- File > Scripts > Load Files Into Stack
- Select all layers and change mode to lighten

Ok so, my first caveat, Photoshop does NOT like 14000 + images

Is there an ImageMagick way of achieving the desired effect above?

I tried successfully using Image Magick creating a PSD file using only 20 images, then opened that in Photoshop and changed the blend mode of the layers to Lighten and got my desired image but have not even contemplated trying to make a PSD with 14,000 layers as I am confident photoshop will crash trying to open it.

So while I can create the PSD, I need Imagemagick to perform it's magic with all those layers in the PSD and then create me a flattened JPG using the supplied PSD.

I will keep trying for now but hope someone can assist me.

[snibgo]

What version of IM on what platform?

Are your 14,495 inputs an aerial view of a continent, or what? How large are they (how many pixels)? Can you link to a couple of samples?

If your inputs are, for example, *.png then:

Code: Select all

magick *.png -compose Lighten -layers Flatten out.png

But if your inputs are too large to fit into memory at the same time, you could do it in a shell "for" loop. If you need better performance, write a script that writes an IM script, so magick is run only once and you don't need temporary files.

EDIT: I forgot to put "-" at the front of "-layers".

[MitchellK]

Many thanks for the reply @snibgo

My version is:

Code: Select all

Version: ImageMagick 6.8.9-9 Q16 x86_64 2018-06-11 http://www.imagemagick.org (Ubuntu 16.04)

All images are jpg, exactly the same map saved every 11 minutes of the day and all sized at 600 px x 530 px

I did try

Code: Select all

convert -compose Lighten layers Flatten result.jpg

and not getting the desired result

my difference is I'm working with JPEG's , should I use mogrify to make them all PNG first?
I also am using

Code: Select all

convert

and don't have any package

Code: Select all

magick

Memory is becoming an issue so I will definitely need to look at a bash loop for it.

This is my output of

Code: Select all

convert -compose Lighten layers Flatten result.jpg

Code: Select all

convert 2018-04*.jpg -compose Lighten layers Flatten aaaaaaaa.jpg
convert: unable to open image `layers': No such file or directory @ error/blob.c/OpenBlob/2712.
convert: no decode delegate for this image format `' @ error/constitute.c/ReadImage/501.
convert: unable to open image `Flatten': No such file or directory @ error/blob.c/OpenBlob/2712.
convert: no decode delegate for this image format `' @ error/constitute.c/ReadImage/501.

I've placed 10 sample JPEG's and a desired-result image here: https://www.dropbox.com/sh/mte4uhx0cjrq ... WI1ha?dl=0

[snibgo]

Sorry, I forgot the "-" in front of "-layers", and you also need "-background None". And your attempts need an input, eg "2018-04-01*.jpg".

Code: Select all

convert 2018-04-01*.jpg -background None -compose Lighten -layers Flatten out.png

[MitchellK]

Oh awesome thank you I will give this a whirl first thing in the morning and get back to you. Thank you so much for your time.

[snibgo]

My command reads all the inputs, then processes them. Assuming IM V6 Q16, this needs 36 GB of memory for all 14495 inputs.

If you don't have that much free memory, you can combine them one by one like this (bash syntax):

Code: Select all

convert \
2018-04-01_00-00-LIGHTNING-SA.jpg \
-background None -compose Lighten \
2018-04-01_00-11-LIGHTNING-SA.jpg -layers Flatten \
2018-04-01_00-22-LIGHTNING-SA.jpg -layers Flatten \
2018-04-01_00-33-LIGHTNING-SA.jpg -layers Flatten \
2018-04-01_00-44-LIGHTNING-SA.jpg -layers Flatten \
2018-04-01_00-55-LIGHTNING-SA.jpg -layers Flatten \
2018-04-01_01-00-LIGHTNING-SA.jpg -layers Flatten \
2018-04-01_01-11-LIGHTNING-SA.jpg -layers Flatten \
2018-04-01_01-22-LIGHTNING-SA.jpg -layers Flatten \
2018-04-01_01-33-LIGHTNING-SA.jpg -layers Flatten \
out.png

This needs memory for only two images instead of 14000 images.

You can build that command by creating an environment variable, or writing a script.

[MitchellK]

Thank you soooo much @snibgo it works perfectly
I did a test on one day's worth of images and then a whole month.

The one day is fast and of course the 1 month takes much longer and puts pressure on the machine, so for now my workflow will be to generate them day by day and at the end of the month take the 30 days worth of already combined images and then just recombine them using the same command line which should take mere seconds.

Code: Select all

time convert 2018-04*.jpg -background None -compose Lighten -layers Flatten month.png

real	20m9.796s
user	4m20.236s
sys	0m35.564s

Code: Select all

time convert 2018-04-01*.jpg -background None -compose Lighten -layers Flatten day.png

real	0m10.809s
user	0m3.352s
sys	0m0.144s