Overlaying two images Magick++

[HunteX]

Hi! I have 2 images:

first:
http://img696.imageshack.us/img696/1261/rd1mz.jpg
second:
http://img705.imageshack.us/img705/8807/traei.png

How i can overlay these images and get this image:


In OpenCV I do that so:

Code: Select all

...
// create mask
IplImage *msk = cvCreateImage(cvSize(firstImage->width, firstImage->height), 8, 1);
// convert mask to Grayscale
cvCvtColor(firstImage, msk, CV_BGR2GRAY);
...
cvCopy(secondImage, resultImage); // copy second image to result image
...
cvCopy(firstImage, resultImage, msk);  // copy first image to result image with mask

How do that with magick++ ?

p.s. black pixels in the second image are RGB (0,0,0)

[anthony]

The whole sequence is known as 'panorama overlaying' and involves a number of steps

• fix any lens distortion in the images
• Locate common 'registration points' between the two images
• distort images using perspective into a shared distorted space (typically one image is left undistorted)

Note that images are also generally transparency graded so as to remove the hard overlap edge that you have.

At this time IM can handle all aspects (as individual steps, not as a overall process) except 'registration points'.

However once you have registration points you can use perspective image distortion with those points to map the images.
You can use more than four points in perspective distortion, and IM will generate a least-squares average of the points so as to produce which should be a closer approximation of a perfect fit.

WARNING: coordinates must be in image coordinates (top-left edge=0,0), not pixel coordinates (centre of top-left pixel=0,0)

[HunteX]

The whole sequence is known as 'panorama overlaying' and involves a number of steps

• fix any lens distortion in the images
• Locate common 'registration points' between the two images
• distort images using perspective into a shared distorted space (typically one image is left undistorted)

Note that images are also generally transparency graded so as to remove the hard overlap edge that you have.

At this time IM can handle all aspects (as individual steps, not as a overall process) except 'registration points'.

However once you have registration points you can use perspective image distortion with those points to map the images.
You can use more than four points in perspective distortion, and IM will generate a least-squares average of the points so as to produce which should be a closer approximation of a perfect fit.

WARNING: coordinates must be in image coordinates (top-left edge=0,0), not pixel coordinates (centre of top-left pixel=0,0)

Thanks for the info!
I've done some of the steps (and the above shows the result of my program implemented in OpenCV).
1. I get the key points by SURF
2. Calculate a transformation matrix
3. Perform the image transformation

I asked the question wrong:

I just need to put (merge) one image to another, using a bit mask. (see the example code above).
Here's how I imagine the algorithm:
1. Getting bit mask from the second image
2. Create the main image
3. Copy the first image in the main
4. Copy the second image in the main with the previously received bit mask

Maybe I can do differently, or easier, but I don't know how to do this with ImageMagick

Why I chose ImageMagick? Because I noticed a lot of interesting opportunities, as well as IM can work with large images.

p.s. If you noticed, sorry for my bad English

[anthony]

No need for a bitmask, just turn on alpha, and -virtual-pixels transparent, then do the distort, the resulting alpha then is the bitmask! and the distorted image can be just overlaid (using -layers merge)

Code: Select all

convert image2.png  -alpha set -virtual-pixel transparent \
         +distort Perspective '@coordinates_file.csv' \
         image1.jpg +swap  -background transparent -layers merge \
        +repage result.png

The coordinates_file.csv is a file of the of the form

Code: Select all

   u1,v1,x1,y1
   u2,v2,x2,y2
   u3,v3,x3,y3
   ...

where u1,v1 is the image coordinate (0,0 = edge) in the second image and x1,y1 the corresponding coordinate in the first image (you are distorting too). To convert a pixel coordinate to image coordinate add 0.5 to all values.

If you like to share the coordinates from SURF (and perhaps some more detail on how to use SURF) I (and others) can try out the above and check results. If you OKAY it I will even convert the above into a actual example in IM Examples (with your name as contributor if you want it).


If you have a transformation matrix already, the you can use PerspectiveProjection for the distortion
http://www.imagemagick.org/Usage/distor ... projection
However you will have to verify the 8 coefficients are in the right order.

Do you have a web link to the appropriate SURF documentation? I don't know that program, so I don't know the meaning of the values you talk about.

[HunteX]

Code: Select all

convert image2.png  -alpha set -virtual-pixel transparent \
         +distort Perspective '@coordinates_file.csv' \
         image1.jpg +swap  -background transparent -layers merge \
        +repage result.png

How I can do this using Magick++ ? What functions need use?

I found the following function:

Code: Select all

Image.virtualPixelMethod(VirtualPixelMethod::TransparentVirtualPixelMethod); // -virtual-pixel transparent ?
Image.distort(DistortImageMethod::PerspectiveProjectionDistortion, params, matrix, true); // +distort ?

That's all

SURF (Speeded Up Robust Features) is a robust image detector & descriptor, first presented by Herbert Bay et al. in 2006, that can be used in computer vision tasks like object recognition or 3D reconstruction. It is partly inspired by the SIFT descriptor. The standard version of SURF is several times faster than SIFT and claimed by its authors to be more robust against different image transformations than SIFT. SURF is based on sums of approximated 2D Haar wavelet responses and makes an efficient use of integral images.

It uses an integer approximation to the determinant of Hessian blob detector, which can be computed extremely fast with an integral image (3 integer operations). For features, it uses the sum of the Haar wavelet response around the point of interest. Again, these can be computed with the aid of the integral image.

The algorithm is patented in the US

from http://en.wikipedia.org/wiki/SURF

I'll give part of my presentation:



After that, we have many pairs (filtered by RANSAC) of keypoints on each image
Next, we can get (calculate?) transformation matrix (based on keypoints)

[anthony]

Code: Select all

convert image2.png  -alpha set -virtual-pixel transparent \
         +distort Perspective '@coordinates_file.csv' \
         image1.jpg +swap  -background transparent -layers merge \
        +repage result.png

How I can do this using Magick++ ? What functions need use?

I found the following function:

Code: Select all

Image.virtualPixelMethod(VirtualPixelMethod::TransparentVirtualPixelMethod); // -virtual-pixel transparent ?
Image.distort(DistortImageMethod::PerspectiveProjectionDistortion, params, matrix, true); // +distort ?

I am not real familar with Magick++ as I program in C and in MagickCore, with Command line interface (yes I am old fashioned), but...
the params is number of parameters, 8 for PerspectiveProjectionDistortion
matrix is a floating point array of those paramaters. true means + form of distort.
There should be some Alpha method to ensure the alpha channel is enabled in the image before the distortion.

I'll give part of my presentation: ....
After that, we have many pairs (filtered by RANSAC) of keypoints on each image
Next, we can get (calculate?) transformation matrix (based on keypoints)

What exact program are you using?
Remember I am Dumb when it comes to Windows, having grown up using UNIX command line!
I hate the limitations and restrictions inherent in a fixed GUI program, prefering to work more
directly with images and image data, and only using GUI for display purposes.

Can you send me the list of the coordinates you found in the first two test images?
As well as the matrix values you determined, so I can check IM internal generated matrix against it.
At least then I can experiment even if I can't generate them myself (Yet).

[HunteX]

What exact program are you using?
Remember I am Dumb when it comes to Windows, having grown up using UNIX command line!
I hate the limitations and restrictions inherent in a fixed GUI program, prefering to work more
directly with images and image data, and only using GUI for display purposes.

Can you send me the list of the coordinates you found in the first two test images?
As well as the matrix values you determined, so I can check IM internal generated matrix against it.
At least then I can experiment even if I can't generate them myself (Yet).

I am using OpenCV and GDAL. This is a transformation matrix:

Code: Select all

	matrix[0] = 0.7635364058950651;	matrix[1] = 0.3438769700704157;	matrix[2] = 56.1722052397604020;
	matrix[3] = -0.4476195206801218;	matrix[4] = 0.8427856192168595;	matrix[5] = 79.1955261689579830;
	matrix[6] = -0.0003992035432886;	matrix[7] = -0.0000913103642461;	matrix[8] = 1.0000000000000000;

But I can't show you the coordinates of the 'filtered' points on both images, since the OpenCV function (cvFindHomography()) does not modify 'unfiltered' points, it only returns the transformation matrix. I can show 'unfiltered' points on both images (if necessary)...

I used the following command to convert and merge the images:
first image | second image

Code: Select all

convert rd2.jpg  -alpha set -virtual-pixel transparent -distort PerspectiveProjection "0.7635364058950651, 0.3438769700704157, 56.1722052397604020, -0.4476195206801218, 0.8427856192168595, 79.1955261689579830, -0.0003992035432886, -0.0000913103642461" rd1.jpg +swap -background transparent -layers merge +repage result.jpg

But image is not expanded, remained equal to the size of the first:

I am not real familar with Magick++ as I program in C and in MagickCore, with Command line interface (yes I am old fashioned), but...

Who can help me with the Magick++ ?

[anthony]

You must use +distort so that that distort will calculate an appropriate canvas size and virtual offset for the distorted image.

Hmmm... lit looks to me like the actual location specified by the transformat matrix may also be given as pixel coodinates.
Try adding 0.5 to the matrix values of index 2 and 5 (the big values which are pixel coodinates)

Also save to PNG if you want to preserve the transparency!!! otherwise you get a un-aliased edges on the JPEG image!
To overlay with a different background color change the -background setting before the -layers merge

Code: Select all

convert rd2.jpg  -alpha set -virtual-pixel transparent \
  +distort PerspectiveProjection \
     "0.7635364058950651, 0.3438769700704157, 56.6722052397604020,
     -0.4476195206801218, 0.8427856192168595, 79.6955261689579830,
     -0.0003992035432886, -0.0000913103642461" \
  rd1.jpg +swap -background black -layers merge +repage result.jpg

That seems to line up better, and now matches the previous results.

I think the images still contain some lens distortion as well, as the join is worst in the very corner. That hat will need to be corrected, generally before doing the image registration and matrix determination, for the perspective distortion.


Note if you plan on overlaying more images, leave the background transparent, and remove the +repage (remove virtual canvas offsets from save to intermediate results, then repeat with the next image. A non-lossy PNG or MIFF image file format is a must for repeated processing!


However you can distort each image, and overlay them all in the one command...

Code: Select all

   convert rd1.jpg +alpha set \
               \( rd2.jpg ... +distort .... \) \
               \( rd3.jpg ... +distort .... \) \
               \( rd4.jpg ... +distort .... \) \
               -background {color} -layers merge +repage \
               result.png

[HunteX]

Thank you, Anthony! If I didn't get to do the same thing with Magick++ then do just that