Remove light effects, even illumination

[Cassandra]

Hey,

I'm trying to remove the light effects on pictures like this one to make it an even coloring/illumination. It should be as if it were if photographed without light from front-left.

Important: They all have this same white background.



I found those entries:

viewtopic.php?f=1&t=21906
viewtopic.php?f=1&t=16323

and played around with Fred scripts like redist, retinex or duotonemap, but unfortunately understand too little about image processing to get anywhere.

Can someone help me with my specific problem, that would be much appreciated:)

[snibgo]

The shoe itself might be an even colour. The lighting has given it light and shade, including the shiny highlights and very deep shadows around the base. Do you actually want to remove this light variation, so the shoe becomes a constant colour, like a silhouette?

[Cassandra]

No, only the "white" reflexex that the light causes. I'm not trying to extract a silouette or make the color more even. I really seek only to remove the effects caused by illuminating it from a certain direction.

[fmw42]

I am not sure you can recover the true color from the white reflection as data has been lost without doing some kind of interactive clone painting.

But the simplest way is to use -floodfill or -draw "matte 0,0 floodfill" to make a black and white mask that is black out side and white where the shoe exists. Do not use -fill. Once you have a mask, you can process the image any way that brings out the color best on the white reflection. Then use the mask to put the white background around the outside via composite with the modified and original images or just between the modified image and a white image.

You may need to make another mask of just the white reflection by using -fuzz -fill -opaque. Then process the image to best improve the shoe so the reflection is improved and then mask the improved improved and original together. Then use the outside mask if needed to fix any problems with the outside or use it to subtract the outside from your second mask, so that the second mask only has the white reflection.

I am sorry I do not have time today to help further, but if you need further help, I can try to do some experiments this evening or tomorrow.

Some one else on this forum may be able to run with this concept and help you sooner. That is fine with me.

[snibgo]

I think fmw's approach is sound.

Perhaps the OP could create a version with Gimp or similar, so we can see what what should be removed and what it should be replaced with. Crude approaches are easy (Windows script) ...

Code: Select all

set SRC=shoe.jpg

FOR /F "tokens=1,2" %%i IN ('%IM%identify -ping -format "%%w %%h" %SRC%') DO (
  set WIDTH=%%i
  set HEIGHT=%%j
)

set BROWN=rgb(103,28,25)

"%IMG%convert" ^
  %SRC% ^
  -fill rgb(1,0,0) -fuzz 8%% -floodfill 0x0 White -fuzz 0 ^
  -write x.jpg ^
  -fuzz 60%% ^
  -fill %BROWN% -opaque White ^
  -write y.jpg ^
  -fuzz 0 -fill White -floodfill 0x0 rgb(1,0,0) ^
  shoeD.jpg

... but look horrible. Using Gimp, a much better version is easy by airbrushing the highlights, using colours picked from the dark leather. Importantly, varying the colour within the airbrush retains the realism. Doing this would be possible with IM, but not trivial.

[fmw42]

you could make the glare transparent and use -sparse-color shepards to fill it in. But it would be very slow at this time. see

http://www.imagemagick.org/Usage/canvas/#sparse-color
http://www.imagemagick.org/Usage/canvas/#shepards
http://www.imagemagick.org/Usage/canvas/#sparse_fill

see especially the last link, though without airbrushing or clone painting, I really do not think it will look very good.

If you had a patch of shoe or another image of the same shoe without the glare, then one could mask in the good part from the other shoe or sample to replace where the glare is located.

Again I do not think this will look good, but you can measure the average color of the shoe and mask that in for the glare as well or cut out a section of the shoe that is large enough to fill the glare from another part of the shoe and mask that in. Some blending at the edges may be needed.

[Cassandra]

Maybe I should elaborate on the purpose of my wish:

I have quite a lot of those pictures (different shoes or objects with different lighting) and want to compare them with image feature extraction. It works quite good but some of the features seem to "think" that this kind of light effect is part of the shoe. And the light angle and degree of reflection can be different on different pictures. That's why I'm trying to remove it.

In consequence, the result of any operation does not need to look perfect or ready to publish the resulting picture:) Just good enough to test my hypothesis that removing it makes the comparison better. But of course, the better it works the more likely it is to make the comparison better. But it does not need to look perfect.

But as there are many, I cannot edit them all by hand:) I need a programmatic solution /command line.

@ fmw42 Unfortunately, I also do not have another, non lighted version of pictures. It's not bad if it is slow, I only need to do it once per picture.

@snibgo: Sorry, no Windows anywhere

Thank you very much for you input!

[fmw42]

try this. It floodfills the outside to make it transparent. Then it replaces white (on the shoe) with a color picked off the shoe nearby. Then it resets the background to white. You can adjust the second fuzz value from 50% to remove more of the white, but it will make the rest of the shoe color change a bit.


convert shoe.jpg -fuzz 20% -fill none -draw "matte 0,0 floodfill" \
-fuzz 50% -fill "rgba(91,17,14,1)" -opaque "rgba(255,255,255,1)" \
-background white -flatten result.jpg

[Cassandra]

That looks really good! Sorry I didn't answer earlier, I missed the alert.

I really like making the background opaque and repacing the remaining white.

The remaining challenge is now
a) how to pic the fill color automatically if I have a lot of pictures. Do you have any suggestions?

and b) What can I do if it is a multicolored shoe? Is there any way to replace the white with the bordering color resprectively in several places (not all with the same color). I suspect however this is not the case very often. So just ignore b) unless you have a spontanious idea how to combine a) and b)

Thank you for your help!:)

[fmw42]

You can fill in holes with the boundary of the hole color by using -sparse-color shepards. See http://www.imagemagick.org/Usage/canvas/#sparse_fill and the new (IM 6.8.3.10) sparse-color: setting to replace the txt: and sed commands. See http://www.imagemagick.org/script/formats.php#supported.

You make your image transparent where you do not want to know the color and coordinates, such as the background and glare. Then you can do:

convert image sparse-color:-

and it will list the coordinates and colors of each opaque pixel. You would then have to get the colors converted to rgb (which will probably be in the next release) and do your own average of each channel.

Today you could run my tidbits example separately on each channel of the image to get the channel average. See http://www.fmwconcepts.com/imagemagick/ ... shape_mean

[Cassandra]

Hey Fred,

I tried to follow your instructions:

1. "You make your image transparent where you do not want to know the color and coordinates, such as the background and glare":

I managed to remove all white and create an image with opage on each lighted area:
convert shoe.jpeg -fuzz 30% -transparent white opageShoe.gif
Looks fine:) ( One has watch closely to see it is actually opaque, not white)



2. Replace transparent areas with surrounding colors:
convert opageShoe.gif sparse-color:- | convert opageShoe.gif -sparse-color shepards '@-' result.png

where the first part outputs the opaque pixels. These are then piped into the second command which should replace these transparent pixels with sorrounding colors. However, it does not work:/ What am I doing wrong?

Result of second command:

[fmw42]

two issues

1) you need to make a mask for the outside of the show first, by floodfilling see -draw "matte 0,0 floodfill"

2) you need to disable alpha

convert opageShoe.gif sparse-color:- | convert opageShoe.gif -alpha off -sparse-color shepards '@-' result.png

After the result comes back you need to use the mask to put white back around the outside of the shoe.


I am sorry, I have to run for an appointment. But if you do not figure this out, I will prepare the code for you later this afternoon.

[fmw42]

OK. Here are the commands that work for me.

input:




# create transparent image (but note it makes the outside transparent also) that is why we need the mask
# I also had to increase the fuzz value to remove all slightly whitish border pixels

convert shoey.jpg -fuzz 55% -transparent white shoe_trans.png



# create mask of shoe and its outside

convert shoey.jpg -fuzz 10% -fill none -draw "matte 0,0 floodfill" \
-channel rgba -fill white +opaque none -fill black -opaque none shoe_mask.gif



# process sparse-color the get colors and fill colors
# then put the mask into the alpha channel
# then flatten over white background

convert shoe_trans.png sparse-color:- | \
convert shoe_trans.png -alpha off -sparse-color shepards '@-' \
shoe_mask.gif -alpha off -compose copy_opacity -composite \
-background white -compose over -flatten result.png



NOTE: this result can be improved by blurring the background mask a little so as to anti-alias the boundary between the white background and the shoe.

[Cassandra]

Hey Fred,

ah, mask, got it:)

However, the result really changes a lot of the picture (too much of its original characteristics). Is there a way to only fill the resulting "holes" of white/transparent and not touch the rest?

[fmw42]

Your input image has partially transparent pixels such as near the back of the shoe. It is viewed on light blue background here in the forum. When the final image is flattened against white, that will make it look different than the input when the background is not white. You can view it best by alternating the images in a viewer. See Anthony's flicker_cmp script http://www.imagemagick.org/Usage/scripts/ But you should flatten your input image against a white background first to make a fair comparison.

I do not know how -sparse-color treats partially transparent values. I have assumed it did not change them.

WAIT: This is strange. The input is jpg, which does not support transparency. But when I drag the input image off of my Safari browser I can see through it. I am not sure how or why this is happening. Any way, my statement about partially transparent pixels is false. Nevertheless, use Anthony's flicker_cmp to look for differences.


Note you can keep a little of the highlights in the region of interest by lowering the 55% to 45% or even 45% for the -fuzz XX% -transparent.

Also note that it was not only the mask, but the addition of the -alpha off in the command that allowed the -sparse-color to fill properly.