Application of Jinc to texture filtering?

[BryantMoore]

I'm by no means a programmer, but I've been wondering how well true 2D radially symmetric resamplers like Jinc would benefit texture filtering for applications in 3D graphics such as video games. From what I understand, the fact that Jinc is radially symmetric is what eliminates aliasing over separable methods like bilinear. Bilinear has been used in video games for something like 15 years now. I've done my own tests on enlarged, perspective distorted textures from various games which I have posted a comparison from below. It seems to give a noticeable improvement in quality.

Triangle (Bilinear)


LanczosSharp

[anthony]

yes I can see a big improvement on the full sized images. Mostly with 'blocking' effects. These may be cased by the use of a bilinear interpolated lookup of the texture pattern.

The problem with radially filtered images is that you are working with a 2-d area, rather than fast unscaled interpolation of the nearest 4 pixels, that your results seem to be using. Basically a interpolated lookup is must must faster but produces sever aliasing effects, unless a 'pyramid' type texture mapping scheme is also used to provide pre-scaled pixel lookups (IM can not do this at this time).

You can try this in IM using +filter -interpolate bilinear

This same fact also means that IM should even do better in terms of quality than some of the commercial lens distortion programs out there because of its use of radial filters, rather than interpolated lookup. It just does not have the automatic camera to distort parameter lookup those programs have to make it easy to use.


Note that 'bilinear' is also a type of 'quad mapping' distortion. A Bilinear distortion can be applied using a more specific 2 pass distortion method, just as image resizing can. and that can also produce speed benefits, over a more general distortion that IM is using. But that is not what is being applied, as I can clearly see 'perspective scaling in the resulting images. However bilinear could still be in use but at a finer scale of the texture mapping (doubtful).


Image processing is not tied to 'speed' (which is a game requirement) as much as it is to 'quality', so even when a more general distortion technique with radial filters , as used in imagemagick, produce greatly improved quality.

[fmw42]

Part of the quality improvement may be due to -distort perspective using Elliptically Weighted Averaging. That means that the size and orientation of the region that is resampled is approximated by an ellipse and weighted by various roll-off filters that are chosen by your filter method, such as Gaussian or Lanczoz. The elliptical shape is a better representation of the data needed to avoid aliasing. Anthony can explain further if you need more details or look at http://www.imagemagick.org/Usage/distor ... a_resample

[NicolasRobidoux]

If speed matters (and I imagine it does), I would do EWA filtering with a Keys cubic kernel: They are so simple that they can be evaluated without a look up table: the slowest part of it in the evaluation of each weight is the computation of a square root, which is pretty fast nowadays.

For example, the unnormalized weights for the "sharpest" "accurate" piecewise cubic EWA (discussed at viewtopic.php?f=22&t=19823) can be evaluated as follows once the square of the distance to the relevant pixel has been determined (rsq below):

Code: Select all

if rsq >= 4 then
  w = 0
else
  r = sqrt(rsq)
  if rsq <= 1 then
    w = 1 + ( ((-56*sqrt(2)-27)/46) + ((42*sqrt(2)+3)/46) * r ) * rsq
  else
    rm2 = r + -2
    w = ( (1/2) + ((-14*sqrt(2)-1)/46) * r ) * rm2 * rm2
  endif
endif

which boils down to 1 square root, 3 multiplications and 2 additions in the longest branch. (On some chips it may be better to reorganize the branching differently.)

I can provide similar simple formulas for the other cubic splines.

P.S. Using constant folding and the fact that multiplicative constants don't matter when filtering, I can get rid of one multiplication.

[NicolasRobidoux]

Also, there is some evidence that even LanczosSharp is too blurry for many people's tastes (although some people really like its smoothness). This appears to be the case in your texture maps. I may put out an even sharper one (using the criterion used to derive the RobidouxSharp filter) out soon. It was discussed with Anthony before but dropped B/C being busy.

[fmw42]

Nicolas,

I have been following your threads with interest and hope when you come to some conclusions, you and Anthony will build a table describing which filters are best for which practical situations (upsampling, downsampling, texture, images of humans, checkerboard, rings, factors of two or integer factors, etc). The pros and cons of each also might be useful.

Fred

[NicolasRobidoux]

Fun fact: EWA was invented for texture mapping.

[anthony]

And it is still used in forward mapped distortions to generate 'splat' texturing of 3D objects!