Complex color transfer

[galv]

First, let me thank you Anthony and Fred for the great write-up on http://www.imagemagick.org/Usage/color_mods/.

I want to perform 'color transfer' from a mask1 from image1 to a mask2 from image2.
The end result should look as natural as possible. The shades and details should be preserved.

Example images
================
image1: http://i1184.photobucket.com/albums/z33 ... /lion2.jpg
mask1: http://i1184.photobucket.com/albums/z33 ... 2_mask.png
image2: http://i1184.photobucket.com/albums/z33 ... rilla2.jpg
mask2: http://i1184.photobucket.com/albums/z33 ... 2_mask.png

What I'm thinking is Fred's script, mapcolors, http://www.fmwconcepts.com/imagemagick/ ... /index.php .
Algorithm:
a. Enumerate all colors of mask1 and save to enumcolors_mask1.txt
b. enumerate all colors of mask2 and save to enumcolors_mask2.txt
c. make a CLUT (final_clut.txt) associating one color from enumcolors_mask1.txt to one color from enumcolors_mask2.txt (somehow :S)
d. use mapcolors script with final_clut.txt as argument

However that sounds very inefficient and probably isn't a good way.

Suggestions/Advice/Ideas?

[fmw42]

I am not sure I follow what you are trying to do? Do you want to colorize the image with the colors from the mask or the other way around?

You could try using my histmatch script if you are on Unix/Mac. see link below.

[galv]

I want to colorize the area in mask2 (part of image2) with the colors that exist in mask1 (part of image1). See example pictures.
Does your histmatch script do histogram matching as explained in http://en.wikipedia.org/wiki/Histogram_matching ?

I tried it with the above example images, it gave me a black image with the size of infile2.
$ ./histmatch lion2.jpg gorilla2.jpg colorized_image.png

The following gives me a white image with the size of infile2 again.
$ ./histmatch gorilla2.jpg lion2.jpg colorized_image2.png

What am I doing wrong?

I have IM Version: ImageMagick 6.6.0-4 2010-11-16 Q16 http://www.imagemagick.org

[fmw42]

The script performs histogram matching (like in the Wikipedia page). Several limitations however. First it was not designed for images with transparency. Second, the histogram gets swamped by too much of any color such a black. So it does not perform well on your images. Also you needed to use -c rgb, but I tested your images and could not get them to work, though my examples work fine. At one time I was trying to make a version of the script into something like what you want to do -- recolorize hair. But I never got that working completely and don't recall where I left off on it.

OK. Now I remember. I used my colorspectrum script to generate a spectrum of unique colors for each mask image. Then I used the histmatch script to match the one spectrum to the other. Then I applied the modified spectrum as a lut using -clut to make the changes.

colorspectrum lion2_mask.png lion2_mask_spec.png
colorspectrum gorilla2_mask.png gorilla2_mask_spec.png
histmatch -c rgb lion2_mask_spec.png gorilla2_mask_spec.png lion2gorilla_lut.png
convert gorilla2_mask.png lion2gorilla_lut.png -clut gorilla2_mask_lionized.png


See if that is what you want. I don't guarantee the quality of the results though. But that was the best I could do for something similar I played with a while ago.

[galv]

At the second command I get this error:

$ ./colorspectrum gorilla2_mask.png gorilla2_mask_spec.png
convert: unexpected end-of-file `/tmp/magick-XXFgPE7K' @ error/pnm.c/ReadPNMImage/1285.

Why?

While the commands you posted work, the result is pretty bad.
What you describe is what I want, recolorize hair (or more like color transfer between hair).

What about my idea about mapcolors? Do you think that might work better?

[fmw42]

At the second command I get this error:

$ ./colorspectrum gorilla2_mask.png gorilla2_mask_spec.png
convert: unexpected end-of-file `/tmp/magick-XXFgPE7K' @ error/pnm.c/ReadPNMImage/1285.

Why?

While the commands you posted work, the result is pretty bad.
What you describe is what I want, recolorize hair (or more like color transfer between hair).

What about my idea about mapcolors? Do you think that might work better?

Are you saying that the colorspectrum worked on one file but not the other?
I have no idea what that message is about. You could try changing the dir="." to dir="/tmp" see the line

dir="." # suggestions are dir="." or dir="/tmp"

You also need to have NetPBM on your system. The error message seems to be about NetPBM.

The quality may be limited to the fact that the colorspectrum is from 8-bit color. It might be possible to modify the script to use 16-bit color, but that means the spectrum will be as much 65535 pixels wide. I have not explored that.


I have not had time to think about the mapcolors approach.

[galv]

1. I tried changing that line to dir="/tmp", same thing.

2. I have NetPBM installed:

$ aptitude search netpbm
i A libnetpbm10 - Graphics conversion tools shared libraries
i libnetpbm10-dev - Graphics conversion tools development libraries a
p libnetpbm9 - Shared libraries for netpbm (v9)
p libnetpbm9-dev - Development libraries and header files (v9)
i A netpbm - Graphics conversion tools between image formats
v netpbm-dev

3. This is the resulting gorilla2_mask_lionized.png:


4. I don't think the bad result is originating from the fact that there are not enough colors in the spectrum (8bit). The speckle effect must be something else.
I also tried converting the gorilla mask to grayscale and then manually selecting two colors from the lion mask and then applying the black,white conversion to selected_color1, selected_color2 with +layer-colors. This is what I got. It is smoother than the one above and it has less colors. But still it isn't good. Doesn't seem natural.


5.

Are you saying that the colorspectrum worked on one file but not the other?

It worked on both, meaning that it produced the spectrum images on both. But on the second one I get this error message.

[fmw42]

I would agree that using +level-colors looks much better.

You could take that one step further and use 3 colors with my tricolorize script. Perhaps the mean color and two extremes (in hue? or intensity?). Try playing around with that.

Your concept of mapcolors or atleast the clut is just what I was doing with colorspectrum. It gets all the unique colors in each image. The histmatch then tries to associate them to create a lut for -clut. So unless you have some manual way to associate colors from each "spectrum" or txt: format file, I don't see anything by using mapcolors.

I don't think this will make a difference but one might try sorting the colors by hue in the spectrum before doing the histmatch.

Here are some results from my old tests.

Original Blonde:


Original Brown:


Recolored Blonde to Brown:


Recolored Brown to Blonde:


I was not real happy with the results, either. That is why I never made a formal script out of it.

[fmw42]

try this (smoothing the spectrum)

colorspectrum lion2_mask.png lion2_mask_spec.png

convert gorilla2_mask_spec.png -filter triangle -resize 10x100% -interpolate bicubic -resize 1000x100% gorilla2_mask_spec2.png

colorspectrum gorilla2_mask.png gorilla2_mask_spec.png

convert lion2_mask_spec.png -filter triangle -resize 10x100% -interpolate bicubic -resize 1000x100% lion2_mask_spec2.png

histmatch -c rgb lion2_mask_spec2.png gorilla2_mask_spec2.png lion2gorilla_lut2.png

convert gorilla2_mask.png lion2gorilla_lut2.png -clut gorilla2_mask_lionized2.png

I get:



You can play with the smoothing by adjusting the width percent in the two resizes. I used a factor of 10% and then 1000%. But perhaps that is too smooth.

Or try converting to grayscale and using my tricolorize script as mentioned above.

[galv]

Thank you for your replies Fred.

Are you sure that you used exactly these commands to get those results on your hair images? I tried the commands on your sample images and I get a totally different result!

I also installed the latest IM version from source just to see if the NetPBM error would get away (and if that was causing such a different result).
Now I have this version and the NetPBM error disappeared:

Version: ImageMagick 6.7.0-10 2011-07-04 Q16 http://www.imagemagick.org
Copyright: Copyright (C) 1999-2011 ImageMagick Studio LLC
Features: OpenMP HDRI

The exact commands I used were:

Code: Select all

./colorspectrum blonde.png blonde_mask_spec.png
./colorspectrum brown.png brown_mask_spec.png
./histmatch -c rgb blonde_mask_spec.png brown_mask_spec.png hairlut.png 
convert blonde.png hairlut.png -clut hair2.png

I got blonde.png and brown.png from your images posted above.

hair2.png (recolored blonde to brown) :


Can you please try these commands as well and tell me if you're getting a different result?

[fmw42]

I just posted the images that I had used from some time ago and don't recall the exact commands. But I will try again with these and let you know.


you have your images reversed

./histmatch -c rgb blonde_mask_spec.png brown_mask_spec.png hairlut.png

should be

./histmatch -c rgb brown_mask_spec.png blonde_mask_spec.png hairlut.png


However, I am getting similar speckling to your image. I will have to look into this further. Unfortunately, I am not sure I know exactly what I did or if there is some error here now in one of the scripts.

I tested using my original lut and get similar speckling. Perhaps there has been some change to -clut?

I also tried using -fx in place of -clut, and get the same speckling.

I looked into my old email messages on this topic and the above is exactly what I was doing and I am getting all the same spectrum images and histmatch results. But the -clut is not turning out as it used to. I am not sure what is going on or what has changed or if my old notes are missing something.

[fmw42]

I reinstalled IM 6.5.8.10 from about the time of my first tests in 2009 and tried again. But still getting the speckling effect. So I am now more puzzled what I may have done differently.

[fmw42]

I figured out what I did to make it work. Rather than using the matched spectrum from histmatch as the lut for -clut, I actually output the lut from with histmatch that did the matching. Then I used that lut on image.

If you email me at fmw at alink dot net, I can email you back with a temporary modified histmatch script that will save the internal lut as histmatch_lut.png. Then you can use that to apply to the image. For example:


histmatch1 -c rgb brown_spec.png blonde_spec.png null:

will produce an image called histmatch_lut.png which I then renamed as blonde2brown_histmatch_lut.png

Then I apply that to the blonde image as:

convert blonde.png histmatch_blonde2brown_lut.png -clut blonde2brown_histmatch_lut.png

That reproduces my blonde2brown result above.

One may have to filter out certain stray colors from the spectrum or from your original image (for example there is some stray green in your gorilla mask image). I can help you with that if this technique proves useful.

Fred