future of image enlargement?

[NicolasRobidoux]

Fred Weinhaus sometimes asks what I think the best ways to enlarge are. And I generally answer that I'm working on it and will let him know when I figure it out.

This being said, it would not surprise me if it's something more or less like this: http://www.wisdom.weizmann.ac.il/~visio ... .html#comp.

P.S. On second look, not quite as artifact free as I initially thought, and not quite the approach I think may be "it".

[NicolasRobidoux]

... although on some of these test images, my own LBB-Nohalo method (Upsharp in NIP2, Lohalo in GEGL, Nohalo in VIPS) does quite well.
And I would think some of the IM filters would do well too.
(Received a pointer to this half an hour ago from John Cupitt...)

[NicolasRobidoux]

... on some others (those that were produced with sharpening, which is the LBB-Nohalo weak spot?), LBB-Nohalo is too soft and a bit too jaggy. I bet one of the EWAs, or even standard Lanczos, could be tweaked to do well.

In other words: The method described in the top link looks very promising, and uses really smart ideas, but I'm not sure at this point that it is the cat's pajamas. Maybe. Maybe not.

[NicolasRobidoux]

Actually, probably not the cat's pajamas: ImageMagick's own EWA LanczosSharp "through sigmoidal" has less artifacts, and I bet adding a bit of tight radius sharpening would produce a similar sharp look without some of the clearly artificial stuff. (The superresolution fractal stuff kind often has the "Jensen filtered look", too.)

But I really liked the idea.

And yes, I kinda expect that some sort of neural network will one day produce the mother of all schemes. But I tend to be a pessimist (I don't do neural networks) so maybe my methods will survive WW3.

[NicolasRobidoux]

I had read the abstract of the article a bit too fast: There is a strong fractal component to the method. Unsurprisingly, it does extremely well at resizing features that appear in original image at multiple scales. An example of such an image is an eye chart. What's re-enlarged is a downsample of a crop of http://en.wikipedia.org/wiki/File:Eye-chart.jpg:

"SR" does amazingly well on this test image: Scroll down until a bit past 1/3 of http://www.wisdom.weizmann.ac.il/~visio ... .html#comp and you find

This is the usual "fractal" miracle, actually.
Compare with my LBB-Nohalo method for example

Code: Select all

vips im_affinei input_small.png nohalo.png nohalo 3.018691589 0 0 3.014925373 0 0 0 0 324 405

(I've not made 100% sure to preserve the alignment and aspect ratio: quick and dirty libvips command):

Although SR has slight "JPG-y" artifacts, it wins hands down.
Also compare with my current work in progress, sigmoidal EWA LanczosSharp

Code: Select all

magick input_small.png -set colorspace sRGB -colorspace RGB +sigmoidal-contrast 10,80% -filter LanczosSharp -distort resize x405 -sigmoidal-contrast 10,80% -colorspace sRGB lanczossharpLinearTenEighty.png

Finally, sigmoidal "regular" Lanczos

Code: Select all

magick input_small.png -set colorspace sRGB -colorspace RGB +sigmoidal-contrast 10,80% -filter Lanczos -resize x405 -sigmoidal-contrast 10,80% -colorspace sRGB RegularLanczosLinearTenEighty.png

and plain regular lanczos

Code: Select all

magick input_small.png -filter Lanczos -resize x405 RegularLanczos.png

P.S. Maybe I'm overdoing the "sigmoidalization". 10 is quite strong.

[fmw42]

try with Catmull-ROM and Lagrangian. They seem to do well (keep things sharper) when enlarging.

[anthony]

It seems strange that they (SR website) used a very blurry bi-cubic for most of their 'comparison' images.
Actually for the first image I rather Kim over their version.

I also noticed that they did not use any image with 'morie' effects!

From this I would say their test images and examples are 'biased' in favor of their algorithm.

Still the enlargement of the Eye Chart image is impressive. Eye charts seem to make good test images for sharpness, without becoming overlay large for display.

Actually the above results impressed on my how good enlargement with sigmoidal contrast is. Particularity the 'T' on the lowest line, which few recovered well. I would think a OCR program on that image would be able to figure out that 'T' on that image, but fail with all other images (including the original).

How well an OCR program recovers the text may make a nice quantitative result, rather than the typical qualitative results we usually have to rely on.

[NicolasRobidoux]

Here is the result of my current pet project, EWA sigmoidal lanczos 3 with deblur set so that the radius of the EWA disk equals the number of lobes:

Code: Select all

magick input_small.png -set colorspace sRGB -colorspace RGB +sigmoidal-contrast 10,80% -filter Lanczos -define filter:blur=.9264075766146068 -distort resize 328x -sigmoidal-contrast 10,80% -colorspace sRGB lanczosRadius3TenEightyLinear.png

[NicolasRobidoux]

Anthony: I've not had time to do a full comparison, but in the case of human faces, I personnally prefer the results with some of my favorite schemes over those with the SR method.
They look more natural.
My EWA schemes are put together for "natural images". That they, as well as standard orthogonal lanczos 3, do fairly well with text when used through my favorite "sigmoidal colourspace" was a pleasant surprise.

[NicolasRobidoux]

I think that using a midpoint other than 50% with linear light may not always be best: It will look better when the image is dominated by dark objects on light foreground (which is often the case; here, for example), but it will not if the objects are light over dark. I am considering sticking to 50% in "light" of the worst case scenario, and to maintain light/dark balance for binary images.
The only thing that makes me unsure besides the fact that a midpoint of 80% in linear light looks good, is that our eyes are more sensitive to small dark variations than small light variations. From a psychovisual viewpoint, it may not be best to treat 0 and 255 symmetrically.

[NicolasRobidoux]

Here is the result with a "safe" all around EWA Jinc-windowed Jinc sigmoidal though linear light setting also discussed at viewtopic.php?f=22&t=21415&start=30#p87744:

Code: Select all

magick input_small.png -set colorspace sRGB -colorspace RGB +sigmoidal-contrast 6,50% -filter Lanczos -define filter:blur=.9264075766146068 -distort resize 328x -sigmoidal-contrast 6,50% -colorspace sRGB lanczosRadius3SixFiftyLinear.png

As suggested earlier, I believe that one will often get better results by moving the sigmoidal midpoint up, which has the indirect effect of "dilating" dark on light features. But this is not a "safe all around" set up, I believe. Black/white symmetry is important.
I've also reduced the contrast value from my earlier 10, because sometimes I believe that it leads to small local colour artifacts. 6 appears to be "safe".
Again, I'm not really sure (and I don't have the energy to fully program a partial "check" that the new methods are "perfectly tuned"), but this looks pretty good using the eyeball metric.
(Note: As some of you know, I try to balance sharpness with artifacts instead of going all out for sharpness. This result is a pretty balanced on IMHO.)

[NicolasRobidoux]

I tried another test image from the superresolution people, and get an amazing result.

Code: Select all

magick input_small.png -set colorspace sRGB -colorspace RGB +sigmoidal-contrast 15,50% -filter Lanczos -define filter:blur=.9264075766146068 -distort resize 400% -sigmoidal-contrast 15,50% -colorspace sRGB lanczosRadius3FifteenFiftyLinear.png

This is a 4x enlargement of

[NicolasRobidoux]

I think that my worries about high contrast leading to colour drift were just that?
P.S. No, they are not: It depends on the sharpness of the input image.

[henrywho]

I think that my worries about high contrast leading to colour drift were just that?
P.S. No, they are not: It depends on the sharpness of the input image.

Let me try Avisynth+NNEDI3+EEDI3 tonight.

[NicolasRobidoux]

...
Let me try Avisynth+NNEDI3+EEDI3 tonight.

Thank you Henry.
Note: If what you are checking is downsampling, note that I don't expect sigmoidal to help. (But I've not checked, and I HOPE it helps.)
For one thing, haloing is much less of an issue when downsampling.
Also: Use 7,50% from now on. I'm pretty sure that within integer contrasts, this is the best overall. (I'm not sure I want to bother figuring out decimal contrasts, e.g. 7.12345.)
P.S. I would not be surprised if the "future of image enlargement" is something not unlike NNEDI3. Not that I don't have reservations about this scheme. But this overall approach has potential for greatness.