Tilt Shift

[gritter]

Hi guys,

I am trying to produce a tilt-shift effect in ImageMagick via the command line but I am having some problems.

My set up:
ImageMagick-6.7.6-1 (Windows) This is a custom build with FFT compiled in, downloaded from here.
Ultimately I will be exec()'ing this from PHP, but it essentially a command-line issue.

The resources:

• Source Image: (800x571)
• The mask: (800x600) I would like to resize the mask to match the source image, in this example it is very close in size.
• The expected result (done in Photoshop): I actually used a lens blur for this image and I understand that a gaussian blur will look different. The important thing to notice is that the image is blurred at the top and the bottom corresponding with the mask image. I think it is important that the mask determines the strength of the blur; whiter pixels in the mask means a larger blur radius.

Note:- A person may suggest applying a blur to a copy of the source image and applying the mask as transparency, then compositing this over the top of the original image, this does NOT achieve the desired effect.

The Attempt
Running this command

Code: Select all

convert.exe street.jpg ( tiltShift.png -gravity center -resize 800x571"!" -filter quadratic -compose blur -set option:compose:args 11 ) -composite -quality 90% test1.jpg

(exclamation quoted for Windows escaping) give this result:
Superficially it looks very similar to the version created in Photoshop, certainly it could do with a little tweaking, for example the central section of the image is a little blurry, but I'm sure with some fiddling it would be OK.

Certainly I am reasonably happy with this result, but...

The Problem
It is very slow. I mean horribly slow. On my laptop this took roughly 60 seconds to create, Photoshop managed it in a few seconds. It gets worse for larger images since the strength of the blur has to be increased as the size of the image increases. I would be more than willing to sacrifice some quality of the blur if I could make reasonable gains in speed.

Attempted solution
After a lot of digging I stumbled across a method using an FFT to perform a blurring effect on this page, under the section "Blurring An Image - Low Pass Filtering".

This provides the following example:

Code: Select all

convert lena.png -fft \
		\( -size 256x256 xc:black -fill white -draw "point 128,128" \
		-alpha off -blur 0x16 -contrast-stretch 0 -write gaussian16.png \
		-clone 0 -compose multiply -composite \) \
		\( +clone -contrast-stretch 0 -evaluate log 10000 -write lena_gaussian16_spec.png \) \
		-delete 0,3 +swap -ift lena_gaussian16_blur.png

To produce a blurry version of the original image. I downloaded the source image for the example and ran the code on my own machine and my output matched the example perfectly.

So, in order to re-produce the result with my example source image I replaced the image used in the above example with my "street.jpg", fixed the dimensions and tried to produce a blurred image with the following code:

Code: Select all

convert.exe street.jpg -fft ( -size 800x571 xc:black -fill white -draw "point 400,285" -alpha off -blur 0x10 -contrast-stretch 0 -write gaussian.png -clone 0 -compose multiply -composite ) ( +clone -contrast-stretch 0 -evaluate log 10000 -write spec.png ) -delete 0,3 +swap -ift street_blur.png

The output from this is a blank black image.

I have been fiddling with the above example for quite a while, sometimes I manage to produce an image with some faint lines on it, or sometimes even a heavily distorted version of the original, but never something resembling a blur.

The Question(s)

• Broadly, how can I produce a reasonable quality but fast tilt shift effect?
• Is there a faster way to perform my working example, perhaps using some other obscure option to replace my use of blur? Or is this the fastest way to achieve the desired effect in this manner?
• Am I barking up the wrong tree by attempting to use an FFT? Even if I could get it to work could I ever get it to apply the gradient mask properly?

Essentially, since I have been fiddling all day to re-produce something that can be done manually in a fraction of the time, I am more or less done with it. Could somebody come up with a working way of achieving a tilt-shift effect or at least give me a strong pointer?

[Bonzo]

Anthony has an example of tilt-shift on his website - check it out it may be what you are looking for.

[gritter]

Anthony has an example of tilt-shift on his website - check it out it may be what you are looking for.

I should perhaps have cited that as what I based my working example on, it is far too slow.

[Bonzo]

Using Anthony's code on your image on a windows 7 pc took about 4 seconds:

Code: Select all

convert "C:\Users\Administrator\Pictures\street.jpg" -sigmoidal-contrast 15x30% ( +clone -sparse-color Barycentric "0,0 black 0,%h gray80" -solarize 50% -level 50%,0 ) -compose Blur -set option:compose:args 5 -composite "C:\Users\Administrator\Pictures\street_blur.png"

The result is a bit to vivid?

[gritter]

Using Anthony's code on your image on a windows 7 pc took about 4 seconds:

Code: Select all

convert "C:\Users\Administrator\Pictures\street.jpg" -sigmoidal-contrast 15x30% ( +clone -sparse-color Barycentric "0,0 black 0,%h gray80" -solarize 50% -level 50%,0 ) -compose Blur -set option:compose:args 5 -composite "C:\Users\Administrator\Pictures\street_blur.png"

The result is a bit to vivid?

I had a bit of trouble running that command, ImageMagick complained about -sparse-color and I was unable to resolve it. Running the command with -sparse-color removed produced a result that was quite bizarre and unlike the tilt-shift effect that I strive for. I guess the sparse-color argument was important?

Incidentally I ought to point out that I am aware that the speed will be different on different PCs, but for my purposes my working example is far too slow and on my laptop it takes ~60 seconds.

[snibgo]

FWIW, I agree that "-compose blur" is far too slow to be of practical use with real-world photos (eg 5000x7500 pixels). I don't know why or what might be done to improve it. Perhaps a convolution matrix is recalculated for every pixel? It would have to be recalculated as the source mask can be anything. For this particular case, where each horizontal line has constant values, an optimisation is obvious but IM cannot know that.

I think FFT is currently limited to square images.

[fmw42]

I do not see the source image, only the mask and the results provided by the OP. So I cannot generate a result as explained below.

There is no need to use -compose blur to do tiltshift. All you have to do is create a gradient mask, use convert image -blur 0xsigma blurimage to blur the image the maximum amount you want to show. Note that -blur is very quick as it is constant blur and a separable gaussian blur filter. Then use the mask with -compose over to blend the original and blurred image. That should be much faster than using -compose blur, which does pixel variable blurring using a varying ellipse shape determined by the mask.

convert originalimage blurredimage maskimage -compose over -composite result

My tiltshift script at the link below does what I have suggested. I have added some modifications to allow brightness,saturation control of the input and -gamma adjustment of the gradient mask.

[gritter]

I think FFT is currently limited to square images.

If that is the case then that would explain why my attempt at using it failed.

I do not see the source image, only the mask and the results provided by the OP. So I cannot generate a result as explained below.

There is no need to use -compose blur to do tiltshift. All you have to do is create a gradient mask, use convert image -blur 0xsigma blurimage to blur the image the maximum amount you want to show. Note that -blur is very quick as it is constant blur and a separable gaussian blur filter. Then use the mask with -compose over to blend the original and blurred image. That should be much faster than using -compose blur, which does pixel variable blurring using a varying ellipse shape determined by the mask.

convert originalimage blurredimage maskimage -compose over -composite result

My tiltshift script at the link below does what I have suggested. I have added some modifications to allow brightness,saturation control of the input and -gamma adjustment of the gradient mask.

Sorry the source image doesn't show up, here is a direct link to it, though it isn't important, it is just some random image I found to use as an example.

I don't have access to ImageMagick on this machine at this time so I cannot run the example you posted, but, based on your description, I think I understand what you are getting at. Indeed, it appears that it is the first technique that I tried.

The trouble is that I don't like the result; you can clearly see the un-blurred original image showing through in the blurred portions.

I have made an example (in Photoshop) with the two techniques side by side:

direct link

On the left is the technique of blurring an image and then composing it with a mask, on the right is the desired look by controlling blur amount with the same mask.

Note in the left-hand image the tower in the background is clearly visible, only with a smudged halo effect caused by the masked blur, whereas on the right the tower is properly blurred. In both images you can compare the roof of the building on the right-hand side of the image to compare the difference between the two images.

I want to note that I do appreciate everybody's help, and I hope that people aren't put off too much by my dismissal of all the suggestions so far!

I had a thought that maybe I could produce a reduced quality blur effect by (perhaps) simulating a regular blur but only checking every other pixel within the blur radius. This way you would get a blur effect over the same radius but it would be much faster at the expense of quality. I have no idea if such a thing is possible in ImageMagick, or even if the description of my hypothetical technique makes sense. It's just a thought.

[snibgo]

I agree that blending a sharp image with a blurred image is a different effect to a single image with a lesser (or any) blur.

However, there might be mileage in creating two (or more) blurred images. The final image, from the top down, would blend between the greater blur and lesser blur, then the lesser blur to sharp, then sharp to lesser blur, and finally lesser blur to greater blur.

Another thought: a common technique for heavy blurring is to shrink the image, then enlarge it back to the original dimensions. Perhaps the image could be distorted (barrel distortion?) to shrink the areas at the top and bottom, then given the inverse distortion.

[gritter]

However, there might be mileage in creating two (or more) blurred images. The final image, from the top down, would blend between the greater blur and lesser blur, then the lesser blur to sharp, then sharp to lesser blur, and finally lesser blur to greater blur.

I had a similar thought, effectively blurring the image in strips, each strip having a different blur strength. Possibly you would need to have overlapping strips and interpolate between them. I have no idea how to implement this.

Another thought: a common technique for heavy blurring is to shrink the image, then enlarge it back to the original dimensions. Perhaps the image could be distorted (barrel distortion?) to shrink the areas at the top and bottom, then given the inverse distortion.

That seems like a good idea, if I could I would give it a try. Btw, I think it would be "cylinder" distortion.

[fmw42]

Can you identify the steps and values you used in photoshop? I tried to blur in PS using the lens blur and used your mask to composite the blurred and original and cannot reproduce your result?

If on Linux or Mac, my camerablur can do lens defocus via FFT in IM HDRI mode. I can reproduce a similar blurred image to that in PS, but again when blending with your mask, I cannot get close to what you showed for your PS result.


I can produce something like your result using my script, variableblur, with a slight modification. It allows you to select the number of blurred images you want to use and then interpolates between them on a pixel by pixel basis according to the mask. The blur process is fast by using the resizing technique that snibgo mentioned above. The problem is that it is very slow due to the use of -fx to do the linear interpolation. Right now I can see no way to speed it up unless someone wants to code the script or the interpolation into an IM proper function.

Here is my result using 5 blurs and 25 pixel maximum horizontal blur using your image and mask.



This is what I get from Anthony's -compose blur. It is much faster than my script. It is much faster with the 1D blur below than a 2D blur. Time was about 2 sec (with one core on Mac OSX). But the 2D blur looks better though you seem to want the 1D blur look. PS will do things a lot faster. It has been optimized for speed and IM is optimized for quality.


convert street.jpg mask.png -compose Blur -set option:compose:args 25x0 -composite street_comp_blur25.jpg

[anthony]

First I am not certain what command you are using but you seem to be generating a horizontal blur rather than a circular blur for the tilt shift. That isn't right. It may be that the argument handling has broken, so the number is only used as a horizontal blur only, (vertial = 0) rather than for both horizontal and vertical (as it should by default).

That is the difference between the IM and PS results.

Second. The photo is not really a good one for tile shift. Too close to the subject, with too many vertical walls. People in image too close, making feet blurred differently to heads.

Tilt shift make images look like models by making distance and near objects blurly (out of focus).
That works well for a image that is mostly a horizontal plan (viewing into distance, or downward into a city street).
That is the top of image is very far away, and bottom is very close. But tilt-shift goes very wrong for images with lots of vertical walls, or with complex objects that are too close. It fails very badly for 'tunnel' view type images, which this photo is also one. The road is right, but everything else (walls, people, tower, poles, etc) are all wrong.

Vertical poles, towers and people, are as a hole are all the same distance form the camera, and as a result should be all the same amount of blur! That is why this image looks so 'weird' and not model-like when a tilt-shift is applied. The walls of this 'tunnel' needs a vertical 'tilt-shift' effect, rather than a horizontal one.

And finally, No you can not 'blend' a unblurred image with a blured image to generate a tilt-shift. That produces glowing or 'softened' image effect, not a variable blurred image.



The other way to do a model-like focus effect, is to blur the image multiple times and merge different blurs based on distance of object to camera. For example see 'read masked blurs' which blurs the background, completely separately to a foreground (infocus) object.
http://www.imagemagick.org/Usage/masking/#read_mask

Better still take this even further, and separate the image into multiple layers based on distance from camera (using transparency for the masking of each later). This separation is itself a interesting effect when you offset and perspective distort each layer, before re-merging. When all layers are separated, blur each layer as appropriate for distance, then merge (flatten) and remove any remaining transparency. That will produce a near perfect 'blur by distance from camera' result, that should be even better than 'tilt shift' for making model-like images.

This just takes a humongous amount of work! Whcih is why tilt shift is used, but only for specfic types of photos.

[gritter]

Can you identify the steps and values you used in photoshop? I tried to blur in PS using the lens blur and used your mask to composite the blurred and original and cannot reproduce your result?

Absolutely:

Firstly, open the source image and the mask image in Photoshop, copy the mask image to the clipboard and paste it as a new channel on the source image so that the channels look like this:


Then make a selection from the new channel by holding ctrl and clicking on the mask in the channels palette so that you have a selection like this:


Click the RGB channel in the channels palette to view the source image with the selection intact, it will look like this:


Go to Filter > Blur > Lens Blur and apply a lens blur, here I used a radius of 50 pixels:
http://s17.postimg.org/qychj0vvz/step4.png

As I preview this reply it looks like the forum is limiting me to 3 images, so you will have to open that last image yourself.

[gritter]

Second. The photo is not really a good one for tile shift.

Granted. But as I previously stated I am simply shooting for a "proof of concept" of the blurring effect that runs quickly, rather than trying to achieve anything of final quality; but you are right, careful selection is important for the final effect to be convincing at all.

The other way to do a model-like focus effect, is to blur the image multiple times and merge different blurs based on distance of object to camera. For example see 'read masked blurs' which blurs the background, completely separately to a foreground (infocus) object.
http://www.imagemagick.org/Usage/masking/#read_mask

Better still take this even further, and separate the image into multiple layers based on distance from camera (using transparency for the masking of each later). This separation is itself a interesting effect when you offset and perspective distort each layer, before re-merging. When all layers are separated, blur each layer as appropriate for distance, then merge (flatten) and remove any remaining transparency. That will produce a near perfect 'blur by distance from camera' result, that should be even better than 'tilt shift' for making model-like images.

This just takes a humongous amount of work! Whcih is why tilt shift is used, but only for specfic types of photos.

I was thinking of a similar thing with my "strips" idea, but I'm not proposing to go to the lengths of producing masks for each object in the photo in order to blur each one separately (a z-buffer, basically), personally I would be satisfied with the approximation that the imitation tilt-shift effect provides.

[fmw42]

Did you see my comment above that if you use -compose blur and set one dimension to 0, it runs very quickly. I showed a result.

Most of your pictures did not show up. So I am not sure what you did in PS. Nevertheless, as above, you can achieve a similar result in a timely manner with -compose blur