future of image enlargement?

[fmw42]

I was just curious about sharpening compared to using the more sharp lagrangian.

I have not tried comparing EWA vs orthogonal with the resize. But I wonder if using unsharp might work better with EWA as both are using a circular symmetry to do their things.

[NicolasRobidoux]

Still exploring. With good low pass filters, one can really push the contrast up, even with text. There will be jaggies, but they will be smooth and sharp. (Text-like images, really, should get their own method.) Here, I show the chip with the EWA LanczosSharp filter:

Code: Select all

magick input_small.png -set colorspace sRGB -colorspace RGB +sigmoidal-contrast 45,50% -filter lanczossharp -distort resize 400% -sigmoidal-contrast 45,50% -colorspace sRGB LanczossharpFortyfive.png

Contrast=100 is too much with the lanczossharp filter when dealing with text. But not, it appears, 50.
(And yes, sometimes I put -set colorspace sRGB when it's not necessary. Bad habits...)
It's a bit more jaggy on straight diagonal lines, and has a bit more halo, than the amazing NNEDI3, but it has less artifacts on small digits and letters and it is almost as perceptually sharp (even without adding unsharp masking, as suggested by Fred.)
-----
I'm going to leave this alone for a bit. I think that I squeezed just about all the quick and dirty improvement I can from this method.
The only thing I really need to check is that contrast=45 or so does not break some images, in which case I should back it down. But really, there is almost no difference with contrast=15, so I have a lot of backing down "headroom" without changing the visual quality of results with natural images (I think).

[NicolasRobidoux]

I'm not 100% sure, but I think that the current version of sigmoidization will not work well with images which may run out of gamut for any reason, for example which are the result of conversion between colourspaces which produce over/undershoots, or processing which negative/overshot transparency.
Basically, a standard sigmoidal only works well if the data is between two values detemined ahead of time. (I suppose they could be the global min and max, but this is sensitive to one single pixel blips, which I don't like. Or, I suppose, we could pre-clamp, but this messes up negative transparency, or reversibility of colourspace transformations other than sRGB <-> linear RGB with sRGB primaries.)
A robust approach would involve something like a sigmoidal function but which is a monotone bijection between all of the set R of real numbers and itself.
Maybe something like s(x) = x^3 or the signed x^2 or x+x^3/3 (which does have a closed form inverse) where x is the difference from "middle gray" in each channel of a linear light space, together with a scaling.
Given the approximate cubic nonlinearity of the HVS w.r.t. very dark tones, maybe something based on s(x)=x^3?
(I'll move further discussion to the simoidization thread: viewtopic.php?f=22&t=21415.)

[NicolasRobidoux]

Pretty impressive: http://www.ipol.im/pub/algo/g_roussos_d ... erpolation.

[NicolasRobidoux]

Lots of interesting stuff at http://www.getreuer.info/.

[fmw42]

Lots of interesting stuff at http://www.getreuer.info/.

This general interpolation summary is nice -- http://www.ipol.im/pub/algo/g_linear_me ... rpolation/

[NicolasRobidoux]

... general interpolation summary is nice ...

Indeed.

[fmw42]

Here is an interesting experiment. I tried one pass enlargement by 8 on the chip image using -filter catrom and -resize and -distort resize. Then I tried enlarging in steps of two. For the -resize, it did not make much difference. But for the -distort resize, it did produce some smoothing of the jaggies. However, I think I like the -resize results better.

Tests:


convert chip.png -colorspace RGB \
+sigmoidal-contrast 12 -filter catrom -resize 800% -sigmoidal-contrast 12 \
-colorspace sRGB chip_c12_r8.png


convert chip.png -colorspace RGB \
+sigmoidal-contrast 12 -filter catrom -resize 200% -sigmoidal-contrast 12 \
+sigmoidal-contrast 12 -filter catrom -resize 200% -sigmoidal-contrast 12 \
+sigmoidal-contrast 12 -filter catrom -resize 200% -sigmoidal-contrast 12 \
-colorspace sRGB chip_c12_r2x2x2.png


convert chip.png -colorspace RGB \
+sigmoidal-contrast 12 -filter catrom -distort resize 800% -sigmoidal-contrast 12 \
-colorspace sRGB chip_c12_dr8.png


convert chip.png -colorspace RGB \
+sigmoidal-contrast 12 -filter catrom -distort resize 200% -sigmoidal-contrast 12 \
+sigmoidal-contrast 12 -filter catrom -distort resize 200% -sigmoidal-contrast 12 \
+sigmoidal-contrast 12 -filter catrom -distort resize 200% -sigmoidal-contrast 12 \
-colorspace sRGB chip_c12_dr2x2x2.png


The above result seems to be the same without all the extra +-sigmoidal-contrasts as I seem to get the same result when doing:

convert chip.png -colorspace RGB \
+sigmoidal-contrast 12 -filter catrom -distort resize 200% \
-filter catrom -distort resize 200% \
-filter catrom -distort resize 200% -sigmoidal-contrast 12 \
-colorspace sRGB chip_c12_dr2x2x2b.png

[NicolasRobidoux]

@Fred: In exact arithmetic, -sigmoidal-contrast 12 is the exact inverse of +sigmoidal-contrast 12.
And your tests confirm that:
1) sigmoidization does reduce halos; and
2) EWA (-distort) Catrom is horrid when enlarging if there are somewhat sharp interfaces.
The thesis of my student Adam (defending in October) has something to say about 2).

[NicolasRobidoux]

@Fred: I like your results because ... they confirm what I expected.
Repeatedly using Catrom to enlarge in steps, with the IM's "align corners" image geometry convention, is more or less like introducing a little bit of Gaussian blur. When doing it with Catrom, kind of like using a Keys spline which crept toward Mitchell.
And I feel "vindicated" in focusing my attention on 3-lobe windowed methods: I you want maximum sharpness, I believe -resize Ginseng (or Lanczos) is less jaggy. And if don't want jaggies, -distort resize LanczosSharp or LanczosRadius3 are cleaner and sharper.

[NicolasRobidoux]

@Fred: As you probably know, whenever you do tests with -resize methods with overshoots, make sure to use IM in HDRI mode because of the "between orthogonal passes" clipping. It makes a difference, esp. when enlarging a lot.
As you probably know already as well, -distort resize is not affected.

[fmw42]

@Fred: As you probably know, whenever you do tests with -resize methods with overshoots, make sure to use IM in HDRI mode because of the "between orthogonal passes" clipping. It makes a difference, esp. when enlarging a lot.
As you probably know already as well, -distort resize is not affected.

I had not considered that. Thanks for the tip.

[stereoMatching]

The SR result about the alphabets are very good
I could tell what are the alphabets of the last line
even with my eyes without relying on OCR, pretty amazing.

I remember our professor said that fractal base algorithm is patent?
Do that mean we have to pay money if we want to implement
this algorithm even this algorithm would not use by any
commercial purpose?

[jsp]

Our vision lab at the University of Texas develops point prediction algorithms, including enlargement, that use natural scene statistics.

This free site that we recently created allows you to enlarge images using this method:

http://rcm.cps.utexas.edu

We also have a qualitative comparison of this chip.png image with several other methods, including the one in your original post:

http://www.cps.utexas.edu/natural_scene ... pare.shtml

The applications section also has many more examples and comparisons of enlagement (super-resolution):

http://www.cps.utexas.edu/natural_scene ... ions.shtml

Jeff

[NicolasRobidoux]

Jeff:
Nice site. Thank you.
Maybe http://exquires.ca could be of use to someone in your group?