Correcting tile luminance and/or color differences?

[fmw42]

I tried your transfercolor script and it did not seem to have an effect on the tiles.

What was your exact command and can you provide your input images and your output image? Perhaps the statistics of the two images are not different enough to see a change?

[ProGamer]

I tried your transfercolor script and it did not seem to have an effect on the tiles.

What was your exact command and can you provide your input images and your output image? Perhaps the statistics of the two images are not different enough to see a change?

I was trying to transfer the histogram from tile 0, to tile 1, 2, and 3, but that didn't have any visible effect. However, using the histogram from my original unprocessed tiles seems to work extremely well.

I made a comparison album of the different color modes and their effects on the final tiled image: https://imgur.com/a/A1HtN

The tiles I am trying to correct the color on, can be found here: https://imgur.com/a/fXxjO

The original unprocessed tiles can be found here: https://imgur.com/a/TXr5F

I used a variation of this command for each of the 4 tiles in my tests:

Code: Select all

./transfercolor.sh -c ycbcr plane_0_n.png plane_0_o.png plane_0.png

The lab option creates the best tiled output, and the ycbcr option creates the second best option. The RGB color space option seems to create the worst tiled output of the 3.

These results seem interesting as both VGG-16 ILSVRC and Salient Object Detection model were trained on a data set that can be understood in the context of Neural-Style as limiting it to standard RGB images with 8 bits of precision per color channel. In a bit more detail, Neural-Style converts input images to a BGR color space format before it "processes" them, and then it converts them back to the RGB color space before it outputs them. This is because models are trained with the BGR color space. Though as I understand it, the switch between the BGR color space and the RGB color space would likely not be concern with what I am doing? Thought maybe I just need to read up more on the different color spaces as this is expected?


Also, on the Transfercolor script page here: http://www.fmwconcepts.com/imagemagick/ ... /index.php, I noticed what I think might be issues with the description?

You list "hsi" here, but the script does not have an "hsi" color mode option. The other issue is that "rgb" in the script is set as the default, and not "lab" as your webpage says:

-c ... colormode ... colorspace mode of processing; options are: rgb,
..................... hsi or ycbcr; default=lab

[fmw42]

I am confused. You talk about modifying the histogram, but are not using my histmatch script. You seem to be using my transfercolor script.

Sorry about the typos in my script. I have fixed them. They were carryovers from another script that I used as a template for this one. Thanks for notifying me.

What is "Neural-Style"? Is this something your are doing with OpenCV?

There is a transfercolor-like python script that I reference, if you are using python/opencv.

[snibgo]

I assume the four images overlap by 96 pixels.

I adjusted top-left by gain-and-bias so its overlap matches the corresponding area of top-right. Then I joined those by MEBC.

Similarly bottom-left and bottom-right.

Similarly top and bottom, adjusting the top.


I suppose the tiles were made by copying pieces from other images, in the style of the airplane wing, by neural net. Is there a reference to this?

[ProGamer]

I am confused. You talk about modifying the histogram, but are not using my histmatch script. You seem to be using my transfercolor script.

My mistake, I was referring to and using your transfercolor script.

What is "Neural-Style"? Is this something your are doing with OpenCV?
There is a transfercolor-like python script that I reference, if you are using python/opencv.

Neural-Style is a free and open source AI/machine learning project that can capture the style from your "style" image(s) and then it recreates your "content image" in the same style as your style image(s). Neural-Style uses something called, "Artistic style transfer" to transfer your chosen styles to the content image. Artistic style transfer is actually a really new technology and is just over a year old, but things are advancing very rapidly. The original research paper describing artistic style transfer, was published in August 2015 and can be found here: https://arxiv.org/abs/1508.06576

The Neural-Style project can be found here: https://github.com/jcjohnson/neural-style and there is a somewhat extensive wiki here: https://github.com/jcjohnson/neural-style/wiki

As for the tiling script, I am using my bash tiling script here: https://github.com/ProGamerGov/Neural-Tile which in turn uses the neural-style.lua script by feeding it the specified parameters.

[fmw42]

The link to the paper and to https://github.com/jcjohnson/neural-style are very interesting and clever transfer of styles from one image to another. But I imaging it takes lots of computing power or GPU?

[ProGamer]

I assume the four images overlap by 96 pixels.

I adjusted top-left by gain-and-bias so its overlap matches the corresponding area of top-right. Then I joined those by MEBC.

Similarly bottom-left and bottom-right.

Similarly top and bottom, adjusting the top.

It's sort of hard to tell how well the tiles are blended together with the MEBC method, can you please upload a full resolution copy to a site like Imgur? Is this MEBC method possible with bash?

[ProGamer]

The link to the paper and to https://github.com/jcjohnson/neural-style are very interesting and clever transfer of styles from one image to another. But I imaging it takes lots of computing power or GPU?

GPUs are better at the specific kinds of mathematical operations that deep learning uses by orders of magnitude in terms of speed. CPUs are definitely usable, but you need the patience to wait a long time for your results. https://en.wikipedia.org/wiki/Deep_learning

[snibgo]

I've uploaded a jpeg of the full resolution image to http://snibgo.com/imforums/t4123.jpg

The MEBC method uses rectDp.bat, a Windows BAT script that is listed on my "Rectangle boundaries with dark paths" page. It needs a version of IM that is compiled with my "darkestpath" module, which is C code. For that, see my "process modules" page.

EDIT: I should add that, for this task, rectDp.bat is over-complex. The job could be done with a single IM convert command for each input, using darkestpath.

[ProGamer]

Experimenting father with the transfercolor script:

Transfer color from original tile to the new tile using a command like this:

Code: Select all

./transfercolor.sh -c lab plane_0_n.png plane_0_o.png plane_0.png

Then I transfer the color of the normal tiled output image, to the new tiled output image composed of color corrected tiles, like this:

Code: Select all

./transfercolor.sh -c rgb plane_large_final_corrected.png plane_large_final_uncorrected.png large_final_fixed.png

The second color transfer seems to work best with RGB, instead of the LAB color space.

This second step created this output: https://i.imgur.com/jtIoLQD.jpg

Then finally, I apply the classic GIMP Color Enhance and White Balance options to the image:

https://i.imgur.com/Su8R2RD.jpg


For comparison:

The uncorrected tiled output: https://i.imgur.com/JP4UTVW.jpg

And the corrected tiled output using the lab color space: https://i.imgur.com/hUbn8ou.jpg

[fmw42]

I, too, have a number of scripts for color enhancement (see enhancelab for one) and for whitebalancing and autotone correction. I do not know if they will work better than the ones from GIMP

[ProGamer]

So I tried this combination of techniques on another image:

Uncorrected tiled output: https://i.imgur.com/gvhK5VQ.jpg
Corrected tile output: https://i.imgur.com/UedL6gc.jpg
Uncorrected transferred to corrected via RGB: https://i.imgur.com/IOH21D1.jpg

I feel like there is something missing, as while the tiles are now the same color, the 3rd tile's colors are now really faded. GIMP doesn't appear to be able to correct this issue with the tiled output as whole.

For the corrected tile output, I used the lab color space and the original tiles.

I also tested the technique with the YIQ color space (Supposedly good for lumiance transfer) and the YUV color space (Also used by Neural-Style, for keeping the original colors). They don't work well for solving this issue. Is it possible to use the BGR color space in ImageMagick?

[snibgo]

Is it possible to use the BGR color space in ImageMagick?

According to http://stackoverflow.com/questions/3674 ... olor-space etc, BGR or RGB is simply about the order of values within the pixels. It has no impact on the result.

[ProGamer]

So thinking more about how Neural-Style works, and how various GIMP operations affected my output images, I think that better results could be obtained by using the transfercolor.sh script with my style image(s) and the tiles. But the issue that the transfercolor.sh script only accepts one image to copy the color space from (hope that terminology is correct?).

The script would need to be able to take multiple images to transfer the color from like this:

Code: Select all

image1.png,image2.jpg,image3.png

for where to transfer the color from, and the input that you apply the color to would stay the same.

How difficult would this modification of the transfercolor.sh script be?

[fmw42]

If you need more input images, just append them first and use that as the reference image. If you need to apply one image's color to many output images, then you would need to write a script over all the output images.

Code: Select all

convert image1 image2 image3 ... -append miff:- | transfercolor <arguments> - output