Audio equalizer for images?

[JoaCHIP]

Maybe this already exists, and I'm just really bad at using google, but i'll share my little idea anyway.

I deal with audio a lot, and have even coded numerous plugins for that. I've often noticed the similarity between the algorithms used there and what you see in image processing. The most simple example is that "blur" resembles a lowpass filter (attenuates treble) and sharpen resembles boosting the amount of treble.

However, it seems to me that in audio you are presented with much more sophisticated options. A parametric equalizer typically has multiple "bands", and each band has up to three different options: 1) Boost/cut, 2) Frequency, 3) Bandwidth.

Let's say we took this algorithm and converted it into 2D. We would be able to apply EQ to an image, doing some of the tricks that "gaussian blur" and "unsharpen mask" in photoshop currently offers, but in a way more customizable way. Here's a few examples of what would happen, if someone had the time to implement this craziness:

Example 1: Choose to boost by +100% at a high frequency (near nyquist) and a medium bandwidth to reach something that resembles photoshop "sharpen".

Example 2: Choose to boost by -100% at a high frequency (near nyquist) and a medium bandwidth to reach something that resembles photoshop "blur".

Example 3: Choose to boost by +100% at a medium frequency and a wide bandwidth to increase the amount of middle-sized detail in the image (causing a somewhat dreamy old-school looking chemical effect.)

Using several bands of boosts and cuts, you could probably reach all sorts of unusual looking effects. I could also see a few situations where this could be used to solve various problems with the image quality. I've been tempted to trying to write some code to do this, but my coding skills are somewhat mediocre. Has anyone had similar thoughts? Made experiments? Any input is welcome here.

PS: This concept does not really have anything to do with "histogram equalization" btw.

[snibgo]

I suspect you can already do all this. The question is simply working out the equivalents. I'd be interested to hear more.

For example, I often find the difference between an image and its blur, then blur the result. I think this is a smoothed rectified high-pass filter.

Today I'm doing lots of "differences between two blurs", which would be a mid-pass filter.

You can interpolate between an image and its blur, to get an image in between. But if you extrapolate in the opposite direction, the image sharpens -- boosting the frequencies that the blur reduced.

I once needed a quick-and-dirty method of finding one sound clip within another. I converted them both to images and used ImageMagick "-subimage-search".

[JoaCHIP]

Your approach would indeed give a highpass filter, but it gets fairly cumbersome (not to mention slow) if you want to achieve more complex equalization this way. I was thinking of a full-featured parametric equalizer here, which means the following controls (for a 1-band version, that is):

A) Boost or attenuate: An amount from e.g. -1.00 ... +1.00
B) Frequency/size: This could be either a value from 0.0001 to 1.00, ranging from super large blur/sharpen size to the smallest possible blur/sharpen size at nyquist limit, or simply a directly specified size in pixels (would still have to be a floating point value though).
C) Bandwidth in octaves: Small values mean that only a very narrow frequency band is affected (watch out for "ringing" effects), large values would affect a wider frequency area around the frequency chosen by parameter B.

Parameter C would be extremely difficult to achieve using snibgo's method, which would also pose other problems such as an increased risk of highlight/black clipping between steps, decreased precision because of the many iterations needed (Q16 would be quite important here) etc. - That's why I think a dedicated algorithm - based on what is known as a "parametric EQ" in the audio world - would be the way to go. You could really do some better sharpening / blurring effects this way, and also some interesting retro-effects. I also think this could give ImageMagick yet another advantage over the competitors.

I'll see if I can construct some examples if anyone is interested?

[snibgo]

I see what you mean about bandwidth. From what little I know about frequency filtering, there is also a shape parameter -- eg long smooth cutoff, or a sharp cutoff that can introduce ringing. Is that right? I'd be interested in seeing the visual results.

IM can be compiled as Q8 to Q64, with or without floating point. Images are stored in-memory in that format. An iterating algorithm may need floating-point.

I think this could be a very worthwhile addition to IM.

If you compile IM yourself, adding "-process modules" is easy. A module can take whatever arguments you like, and you can do this without changing the factory IM code. However, the interface is at MagickCore level, not MagicWand. Many examples on my page http://im.snibgo.com/customim.htm

[JoaCHIP]

I've made a quick example which only shows a part of what you would be able to do with this, but because I've been using normal image processing algorithms here, I cannot easily recreate steep filters. The "slope" of these filters is most likely 6dB/octave, because they are probably 1. order filter (1-pole filters). For the following filters I'll use two "EQ bands", and on the first and second examples I'll reverse what is boosted and what is attenuated to show some possible variations.

Band A: Boost: -6 dB, frequency = 1.33 pixels, bandwidth: medium wide
Band B: Boost: +6 dB, frequency = 6.67 pixels, bandwidth: medium wide


Band A: Boost: +6 dB, frequency = 1.33 pixels, bandwidth: medium wide
Band B: Boost: -6 dB, frequency = 6.67 pixels, bandwidth: medium wide


Now you are right that "ringing" may occur when designing more unusual filters, especially very steep ones (low bandwidth). For the above examples I've been unable to adjust the bandwidth, because I'm just using Photoshop to simulate these effects. But with a dedicated filter more fancy effects could even be achieved.

[JoaCHIP]

Two more to give some more examples of the effect, now that I'm at it... :

Band A: Boost: -6 dB, frequency = 50 pixels, bandwidth: medium wide
Band B: Boost: +6 dB, frequency = 20 pixels, bandwidth: medium wide
Band C: Boost: -6 dB, frequency = 6 pixels, bandwidth: medium wide


Band A: Boost: -6 dB, frequency = 50 pixels, bandwidth: medium wide
Band B: Boost: +6 dB, frequency = 20 pixels, bandwidth: medium wide
Band C: Boost: -6 dB, frequency = 6 pixels, bandwidth: medium wide

[snibgo]

Coo, excellent! boost20_attentuate4 has a lovely "ethereal" look (when downloaded and viewed full-size). Can you put up a version with no attenuation/boost?

[JoaCHIP]

Sorry, I almost forgot to put up the original...


PS: This image is free for use by anyone, so be my guest!

[Juce]

I did few tests with a raised-cosine filter:



A0 = 1.0;
f0L = 0.02; f0H = 0.05;
A1 = 0.5;
f1L = 0.1; f1H = 0.2;
A2 = 1.0;


A0 = 1.0;
f0L = 0.00; f0H = 0.05;
A1 = 2.0;
f1L = 0.1; f1H = 1.0;
A2 = 0.1;


A0 = 0.0;
f0L = 0.00; f0H = 0.01;
A1 = 1.0;
f1L = 0.1; f1H = 0.5;
A2 = 1.0;


A0 = 1.0;
f0L = 0.0; f0H = 0.25;
A1 = 2.0;
f1L = 1.0; f1H = 1.0;
A2 = 2.0;

[Drazick]

What about doing it using Wavelets / Image Pyramid?

[JoaCHIP]

@Juce: Wow, those are great. Only the third image seems to be affected by the X-Y orientation of the algorithm, but the other examples are great.

Having to type in coefficients directly might scare off most users (and me too, most likely). But I guess it would be a similar operation as calculating coefficients for an audio filter, and creating a UI would resemble creating a UI for a parametriq EQ.

How did you do this? Is there a way in ImageMagick, or did you code this?

[fmw42]

You likely can achieve this by doing highpass, lowpass and bandpass filtering in the frequency domain using FFT processing. Imagemagick has FFT processing. See http://www.fmwconcepts.com/imagemagick/ ... urier.html

You would modulate the magnitude image by a mask that would achieve your gain and attenuation. The mask would be a radial gradient modified by some look up table to increase or decrease the value as a function of distance from the center. The center area is low frequencies and the further out are higher frequencies. The mask would be multiplied by the magnitude image. To get gains greater than 1, you would need to do the processing in HDRI mode.

[JoaCHIP]

fmw42: That's a good idea actually. I could probably just script some stuff on top of that to give me some more intuitive controls, and then my script would generate the necessary imagemagick commands to do FFT --> apply boosts/attenuations (frequency curve response) --> inverseFFT.

I'll report back if I get something to work.

[fmw42]

Here are some simple examples using FFT processing.

Input:


Lowpass filter:

Code: Select all

size=`convert cafe.jpg -format "%wx%h" info:`
dim=`convert cafe.jpg -format "%[fx:max(w,h)]" info:`
convert cafe.jpg -fft \
\( -size ${dim}x${dim} radial-gradient: -evaluate pow 3 \) \
\( -clone 0 -clone 2 -compose multiply -composite \) \
-delete 0,2 +swap -ift -crop ${size}+0+0 +repage cafe_lowpass3.jpg

Radial Profile (across diameter of FFT magnitude):



Highboost filter1 (cutoff is at 80% out along radius to avoid amplifying noise; blur is used to avoid sharp breaks to avoid ringing):

Code: Select all

size=`convert cafe.jpg -format "%wx%h" info:`
dim=`convert cafe.jpg -format "%[fx:max(w,h)]" info:`
im6937hdri convert cafe.jpg -fft \
\( -size ${dim}x${dim} xc:white \
\( -size ${dim}x${dim} radial-gradient: -negate \
-black-threshold 25% -white-threshold 80% -fill black -opaque white \
-fill white +opaque black -blur 0x5 -auto-level \) \
-define compose:clamp=off -compose plus -composite \) \
\( -clone 0 -clone 2 -define compose:clamp=off -compose multiply -composite \) \
-delete 0,2 +swap -ift -crop ${size}+0+0 +repage cafe_highboost1.jpg

Radial Profile (amplitude is 1 and bottom is at 1, so top is at 2):



Highboost filter2 (twice the amplitude):

Code: Select all

size=`convert cafe.jpg -format "%wx%h" info:`
dim=`convert cafe.jpg -format "%[fx:max(w,h)]" info:`
im6937hdri convert cafe.jpg -fft \
\( -size ${dim}x${dim} xc:white \
\( -size ${dim}x${dim} radial-gradient: -negate \
-black-threshold 25% -white-threshold 80% -fill black -opaque white \
-fill white +opaque black -blur 0x5 -auto-level -evaluate multiply 2 \) \
-define compose:clamp=off -compose plus -composite \) \
\( -clone 0 -clone 2 -define compose:clamp=off -compose multiply -composite \) \
-delete 0,2 +swap -ift -crop ${size}+0+0 +repage cafe_highboost2.jpg

Note: using a lookup table applied to the gradient would process faster than processing the gradient directly.

[jmac698]

Just wanted to mention, if you want an equalizer for images try http://www.avisynth.nl/users/vcmohan/FF ... Quiver.htm