undo a composite -dissolve

[anthony]

Some random thoughts

These are always good.
It like a 'box of chocolates' as the saying goes.

the -auto-level works if the overlay contains areas of total transparency and total opacity

I have added a note about this, (now uploaded, should appear real soon).

I have also added a second 'full example' with the simplifications when only black and white backgrounds are involved.

So instead of -auto-level -negate
-fx "u/maxima" -negate
(this command takes rather long to execute, I think imagemagick is calculating maxima for every pixel.

yes it is. I would like to see the -fx handling re-written in many ways. That is one of them. That is some values get cached when they are constant.

Alternatively, I just realized, with your formula:

Sa = 1 - (Rc1 - Rc2)/(Dc1-Dc2)

Code: Select all

convert match_burn_skyblue.png match_burn_gold.png -fx "1 - (u-v)/(u.p{0,0}-v.p{0,0}) match_alpha.png

I should warn you. One of my firsts tests with using u.p{0,0} failed.
I wanted to reduce the number of images, so after I had the alpha, I added it immediatally to the original image. It sort of simplified and removed the needs for extra cloning. I Then tried to use the now fullytransparent p{0,0} color for the recovery step. But the results were all wrong.

Seems that u.p{0,0} returns '0' values for color channels when a pixel was fully transparent, even though non-zero values really are present in the image.

For example...

Code: Select all

convert 'xc:rgba(10%,20%,30%,0)' -fx 'debug(u)' null:
rgba(10%,20%,30%,0)[0,0].red: u=0
rgba(10%,20%,30%,0)[0,0].green: u=0
rgba(10%,20%,30%,0)[0,0].blue: u=0

And yet if I turn off transparency, I get the real values of those channels!

Code: Select all

convert 'xc:rgba(10%,20%,30%,0)' -alpha off -fx 'debug(u)' null:
rgba(10%,20%,30%,0)[0,0].red: u=0.100008
rgba(10%,20%,30%,0)[0,0].green: u=0.2
rgba(10%,20%,30%,0)[0,0].blue: u=0.300008

I have reported this as a bug, as I really did want the fully-transparent color values. Still the 'clone' method I am using seems a little more understandable, even if you end up with four images in memory at one point.

to recover an overlay from a series of images with mixed unknown backgrounds (img001.png-img999.png):

Code: Select all

convert img*.png -evaluate-sequence max maximage.png
convert img*.png -evaluate-sequence min minimage.png

check whether the background area is monochrome in both outputs, and then proceed as above using maximage and minimage as input.

Note -evaluate-sequence would not generate monocrome results. The only reason it does NOT in my use is I separated the channels and use it to get the results from the channel with the 'maximum differences' for any transparency. It still relies on the images being overlayed on solid fixed colors.

In a long sequence such as a video stream (for which logo removal would be a really great feature), may not even generate ALL black and all white, though it is likely you will get a black 'minimum' image, you may not get white. I have been looking for a example for testing purposes. Also know that if the logo is opaque, you may never quite know what was underneath. but at least you can 'hole fill' to remove the opaque parts of the logo.

Still being able to extract the overlay from two 'known' backgrounds, is a great step forward. Basically as two known backgrounds is often easy to get a hold of, especially for 'GIF animations' of highly anti-alised animations you can find on the web.

All I knew was that it should have been possible, as it would provide enough known data to figure out the unknowns of original transparency and color of the overlay. I was just glad I was about to generate a reasonable test image for this purpose.

[anthony]

Now, what if the overlay is in a different location for each image?
It might be possible to align the image and the overlay, but I am unsure what the best method for that would be.

Perhaps an edge-detect and then a subimage search?
Or is it possible to optimize the search with some assumptions?

That should be straight forward. as the two backgrounds are different a sub-image search for maximum correlation should find you the alignment vector. It may not be particularly easy but it should be possible, Ideally using a fast fourier transform to generate a correlation image should do the job.

Remember a image multiply in frequency space (fast fourier transform) generates a convolution.
Rotating one of the images 180 degrees before doing the convolve, generates a cross-correlation image.
That make it ideal for doing sub-image searches with very large images!

See http://www.imagemagick.org/Usage/convolve/#correlate Especially the last part.
I have not done this myself as yet, but Fred Weinhaus (fwm42) has.

[anthony]

Hmmm thinking about known logo removal, I was thinking that rather than leaving the opaque parts of the logo inplace, it may be better to actually make that part a transparent hole.

Code: Select all

   convert  overlaid_image.png    overlay.png    -alpha set \
      \( -clone 0,1 -fx "v.a==1?u:(u-v*v.a)/(1-v.a)" \) -swap 0 +delete \
      -channel A -fx "v.a==1?0:u" \
      overlay_removed_leaving_hole.png

This seems a little awkward to do without using a slow FX, as you really need to invert, and threshold the alpha channel of the overlay image, before using it as a mask to create the hole.

The reason I think a hole is better, is that you could then fill in that hole using the surrounding pixels, rather than leave it as some color unrelated to recovered background. This generally uses the fact that transparent pixels have no 'color' component.

For example -- a blur hole fill...

Code: Select all

     convert  overlay_removed_leaving_hole.png
    \( +clone -channel RGBA -blur 0x3 \) +swap -composite \
    overlay_removed_hole_filled.png

ASIDE: I really have to change -blur to handle alpha BY default, unless channel is set.
(-morphology Convolve does that automatic alpha handling!)

I have a number of hole-filling techniques, which I'll be writing them up into IM examples soon, now that I have a place to put them. Actually this is immediately after the examples for Background Removal using Two Backgrounds.

[anthony]

UPDATE. A simplier method of 'make a hole if overlay opaque!

Just extend the original formula to also use the alpha channel....

Code: Select all

   convert  overlaid_image.png    overlay.png    -alpha set \
      -channel RGBA  -fx "v.a==1?0:(u-v*v.a)/(1-v.a)" \
      overlay_removed_leaving_hole.png

Note the use of '0' for the v.a==1 case

That is a lot simpler! Though I have not tested this if the overlaid image also happens to contain transparency! It does seem to work perfectly fine however for a opaque compound image.


Hmmmm... the FX in this case may be able to be replaced by equivalent and faster image composition methods.

[HugoRune]

I should warn you. One of my firsts tests with using u.p{0,0} failed.
I wanted to reduce the number of images, so after I had the alpha, I added it immediatally to the original image. It sort of simplified and removed the needs for extra cloning. I Then tried to use the now fullytransparent p{0,0} color for the recovery step. But the results were all wrong.

Seems that u.p{0,0} returns '0' values for color channels when a pixel was fully transparent, even though non-zero values really are present in the image.

IIRC that bug did not occcur when I tried this, not sure what I did differently. perhaps I only had black backgrounds

to recover an overlay from a series of images with mixed unknown backgrounds (img001.png-img999.png):

Code: Select all

convert img*.png -evaluate-sequence max maximage.png
convert img*.png -evaluate-sequence min minimage.png

check whether the background area is monochrome in both outputs, and then proceed as above using maximage and minimage as input.

Note -evaluate-sequence would not generate monocrome results. The only reason it does NOT in my use is I separated the channels and use it to get the results from the channel with the 'maximum differences' for any transparency. It still relies on the images being overlayed on solid fixed colors.

What I mean is this: If the darkest color in a sequence of images is "dark gray" or "dark blue" or "black" and you have enough statistically independand images that the darkest color occurs at least once for each pixel over the whole sequence;
then the "-evaluate sequence min" of the whole sequence should be entirely monochrome "dark gray" or "dark blue" or "black" except for the overlay part:

Code: Select all

for /l %a in (0,1,1000) do convert -size 100x100  plasma:fractal -auto-level ^
    match.png -compose dissolve -define compose:args=50 -composite series%a.png

convert series*.png -evaluate-sequence min min.png
convert series*.png -evaluate-sequence max max.png

... -->
(almost. Some irregularities in the top left corner remain)

Even though the match was overlayed with only 50% transparency, and the background is random, it should be possible to recover the overlay almost perfectly.
And since it was overlayed with 50% transparency, the background is then recoverable as well.

Of course, if certain colors are only present in certain parts of the image, then this does not work:

Code: Select all

for /l %a in (0,1,100) do convert -size 100x100  plasma:grey50-grey50 -auto-level ^
    match_recovered.png -compose dissolve -define compose:args=50 -composite series%a.png

convert series*.png -evaluate-sequence min min.png
convert series*.png -evaluate-sequence max max.png

... -->
Seems like plasma:fractal samples the colorspace almost independand of position, while the regular plasma favors certain regions.


Not sure how well that works with images like video screen captures though

[anthony]

the plasma function in IM is has a known bug, but is very deep in the function which would need a lot of time to pull the function apart to find it.

See Problems using Plasma
http://www.imagemagick.org/Usage/canvas ... a_problems

As such I am not surprised you are getting a pattern using it. But then you are only using it for generating test cases.

[HugoRune]

Aside: Some thoughts about removing backgrounds

What if we only have a single source image with a monochrome background?

It is impossible to recover the original alpha channel from this image alone, because there are several possible overlays with different alphas which would produce the same image when flattened over a navy background.

But it is possible and perhaps useful to examine all possible overlays, and select the one with the maximum transparency.
I.e. removing the maximum amount of navy blue from this image.
The resulting overlay will be totally transparent where the original is totally blue, and will be partially transparent where the original contains some amount of navy blue.
And it will not have any blue color spill halo around the flame!

Gimp has a function that does that, it is called "color to alpha" http://docs.gimp.org/en/plug-in-colortoalpha.html

But it is possible with imagemagick too:

This calculates the "maximum possible alpha" (I think. The math is hard)

Code: Select all

convert \
   ( match_navy.gif xc:navy -fx " v==1?0: (u - v) / (1-v) " ) \
   ( match_navy.gif xc:navy -fx " v==0?0: (v - u) / v " ) \
  -separate -evaluate-sequence max  alpha_navy.png

Formula adapted with help from http://gimp.sourcearchive.com/documenta ... ource.html

(It is perhaps easier to understand like this:

Code: Select all

convert \
if u>v:
   ( match_navy.gif xc:navy -fx " v==1?0: (u - v) / (1-v) " ) \
else:
   ( match_navy.gif xc:navy -fx " v==0?0: (v - u) / v " ) \
  -evaluate-sequence max \
  -separate -evaluate-sequence max  alpha_navy.png

)
[/size]
With the alpha the overlay is recovered as in the previous examples:

Code: Select all

convert match_navy.gif alpha_navy.png -alpha Off \
-fx "v==0 ? 0 : u/v - u.p{0,0}/v + u.p{0,0}" \
alpha_navy.png -compose Copy_Opacity -composite rec_navy.png

... ... ... source
... ... ... recovered "maximum" alpha
... ... ... recovered overlay

As can be seen, when projecting these recovered overlays over a vastly different background, the colors will be different. But if you put the match recovered from a particular background over a similarly colored background, the differences are barely noticeable. Since this forum uses a light blue background, the navy and white match look fine, while the gold and black match show strong differences.

But in all cases the flame looks nice and seamless.

For optimal results, the opaque match handle should probably be treated differently from the semi-transparent flame.

[anthony]

Nice exploration of the problem. And it is a real problem.


This is actually the same problem faced by people using 'green/blue screens', and fixing 'color spill'
http://en.wikipedia.org/wiki/Chroma_key
Most of this is about using HSL space to select what is 'green'. The important part is at the bottom of the page.

It is people with blonde or white hair that are suposed to have the worse problem Like the 'flame' in out example.

Of course they have it a little easier in that the object in front is known not to contain any 'near green' colors.

The first step they use is often separate all pixels into three cases. full-transparency, full-opacity
and the problem area.. particial transparency or 'color spill' pixels

The problem is worse in that you have two types of color spill. transparent (alpha is variable)
and reflective (alpha - opaque, just apply a color correction).

Some methods involve creating two shells of colors in colorspace such that inside is green screen
and outside is opaque. but the colors between are color-spill that needs correction. They can get very fancy!

My own notes on color spill (at this time) is...

Code: Select all

 From Wikipedia Chroma Key
    http://en.wikipedia.org/wiki/Chroma_key

      a = K0 * b − K1 * g + K2

    Where K1,K2,K3 are adjustable constants, 1 is good start point

    Spill suppress using...
      g = min(g,b)

From http://discussions.apple.com/thread.jspa?messageID=10937705

  If green is greater than either red or blue
  set it to the average of red and blue!  Else leave it alone.

     g = g>(r+b)/2 ? (r+b)/2 : g

I have NOT tried any of them at this time.

[anthony]

This calculates the "maximum possible alpha" (I think. The math is hard)

Code: Select all

convert \
   ( match_navy.gif xc:navy -fx " v==1?0: (u - v) / (1-v) " ) \
   ( match_navy.gif xc:navy -fx " v==0?0: (v - u) / v " ) \
  -separate -evaluate-sequence max  alpha_navy.png

Formula adapted with help from http://gimp.sourcearchive.com/documenta ... ource.html

With the alpha the overlay is recovered as in the previous examples:

Code: Select all

convert match_navy.gif alpha_navy.png -alpha Off \
-fx "v==0 ? 0 : u/v - u.p{0,0}/v + u.p{0,0}" \
alpha_navy.png -compose Copy_Opacity -composite rec_navy.png

I wonder if it is possible to incorporate a slight fuzz factor for the background. Say a slight level adjustment for the alpha levels (NB: level works with 'matte' values not alpha values).

That is the background does not quite perfectly match the background in the image (JPEG or small patterns in a green screen).

this has two bounds. lower (definatally transparent) and upper (definatally opaque) Of course you will not be able to extract the alpha perfectly, but it should work reasonably well.

Also using difference and divide maths composition could probably be used with this for faster results.