Discussions About Relative Vs Aboslute Displacement

[fmw42]

I have been working to change some of my distortion script that are very slow because they use -fx for rather complicated expressions to the faster method using composite -displace.

I have found that quite a few of my scripts do not comply directly with the assumptions made by the way that composite -displace works.

Anthony and I have been discussing this for some time now and he suggested that I open the discussion to more people before he tries to implement something suitable.

Many of my fx expressions involve evaluating some mathematical expression that is in essence a displacement evaluated at each i,j pixel, BUT applied relative to some fixed point, row or column, which most often is the center of the image (or center row or column). Two examples are my bubblewarp (similar to Photoshop spherize) and my cylinderize scripts. Anthony calls this an absolute displacement.

(Note that with the introduction of the new -function arcsin, arctan and sinusoid, many of my fx expressions can be implemented directly without the need for fx.)

The function composite -displace applies some displacement evaluated at each i,j pixel, BUT applies it relative to the i,j coordinates rather than some fixed coordinate like the image center. Anthony calls this a relative displacement.

Both Anthony and I, independently, discovered that one can modify the (fx or otherwise created) displacement images by subtracting a linear or gradient-like term to effectively convert them for appropriate use by composite -displace.

However, I have suggested that it might be useful to modify composite -displace to have an option to allow it to work relative to some fixed coordinate rather than relative to the i,j pixel. This would avoid the need for the correction to the displacement images. This change is not mathematically complicated. However, it requires a potential change to the arguments for composite displace. For example:

If two arguments are provided, e.g. XSCALExYSCALE, then composite -displace would work as it currently does (a relative displacement) to apply the xscale and yscale to the displacement images and add them to i,j.

However, if four arguments are provided, e.g. XSCALExYSCALE+XREF+YREF, then composite displace would apply the xscale and yscale to the displacement images and add them to xref, yref. This then becomes an absolute displacement.

The issue here seems to be that making such change in the arguments to composite may break other composite functions and would require work to test and to fix them. The other possibility that Anthony has suggested is to rewrite composite -displace so that it can be invoked from convert rather than composite.

Anthony has yet to decide the best approach and we both would like to hear from others about any interest or ideas on these subjects.

Anthony can explain in more detail and much better than I about the software issues.

Please chime in if you have any interest or ideas.

[anthony]

To further complicate this drama, is that I want to get composite displace working in "convert", Either by re-implementing it as a separate operation, or by figuring out some way of providing arguments to the compose setting.

Such arguments would also permit the other few special composite operators, blend, dissolve, to also be directly usable from "convert". Also it would permit the use of -layers composite to displace a single or multiple images, with a single or multiple sequence of displacement operations.

[fmw42]

Anthony,

I had another thought on this as a possible interim solution. Let me know if this makes it simpler or is as hard or more so to code.

Considering that this new feature is a rather mathematically advanced feature, lets consider this somewhat like you did for -distort advanced features, namely, to use -set.

So for example with -distort SRT, you used -set option:distort:viewport 44x44+15+0 to set viewport size and offset.

What if we did the same for composite -diplace XSCALExYSCALE and used:

-set option:compose:displace:referernce +XREF+YREF or
-set option:compose:displace:referernce XREF,YREF or
something like that to provide a flag to convert the relative displacement to an absolute displacement and provide the reference coordinate


Similarly for the -compose MATH that I had asked about as a potential new compose option, we could use:

-set option:compose:math:values A,B,C,D or
something like that

where the math expression is: A*u*v + B*u + C*v +D
where
A=multiply factor for the product of image1 x image2
B=multiply factor for image1
C=multiply factor for image2
D=constant addition amount

or you can rearrange the terms as you like, for example:

where the math expression is: A*u + B*v + C*u*v +D
so that you might want to use:
-set option:compose:math:values AxB+C+D

Any way that you like or makes sense is OK.

(note: D could have a percent added to allow both absolute or percent type additions/subtractions or one can simply limit to absolute values either positive or negative. I can deal with it if limiting to absolute values is the only or simpler way)

Is this use of -set feasible and easier as an interim solution prior to IM 7?

[anthony]

The call structure to CompositeImages() is basically....
source_image, compose_method, destination_image, x_offset, y_offset
Where the x and y offsets has already been 'gravity' modified, if this is required.

However some compose methods need some extra arguments on top of these arguments.
For example....

• 'blend' and 'dissolve' need two 'percentage' arguments (they are always percentages)
• 'displace' also needs two arguments a X and Y displacement vector in terms of pixels.
• 'threshold' (Not availabel form Commandline) also uses two extra arguments of 'amount' and 'threshold' (ranging 0.0 to 1.0) (threshold defaulting to 0.05 if not given.
  Though I can find no use if this composition operator in ANY part of the library.

The current method of handling these 'extra arguments' is to assign the arguments into the 'geometry' meta data (a string) in the 'destination image'. That is the geometry is replaced before the call, with the arguments from '-displace' '-blend' etc... Note that currently "convert" does not do this at all.

There is also a special artifact image meta-data addition 'modify-outside-overlay' which is used when doing 'composite tiling' to disable the clearing of area outside the overlay area (and saving time for which "composite" tiles over the whole image (API function, not a library function).

Basically extra argument handling is a 'hack', in that the extra arguments are passed via the image. But then, traditionally NO extra arguments are needed for image composition!


I am wanting to generalise this extra argument handling, and am starting to feel that passing them as a 'geometry' string is not the best method.

It limits the number of arguments to four (actually five!), (comma separated, or in geometry specification). Actually as 'geometry' is really just a 'string', and I have now determined it is NOT even used to pass X and Y offsets into CompositeImages() it could be used for any number of arguments (as a string, nor an array).

BUT this is still a 'hack' and as such the calling API interface ("composite", "convert", Perl API, etc) all still need to do a lot of data manipulation. Specifically pull out the 'X and Y offsets' from the NORMAL use of the 'geometry' meta data, and replace it with the extra argument data string, if present.

It would be better if such data manipulation was NOT needed. IE geometry is just composite offsets, and a separate meta-data element passes the 'extra arguments' if needed.

"composite" has now problem as it actuall has a separate 'global' data structure to hold the needed information that was saved during option parsing. "convert" does not. The other API's I believe get this data from their API calls.

Before proceeding I need to

• determine how extra arguments should be passed (if we want to improve things)
• check how other API's handle extra arguments

INPUT is very much needed from any and all IM developers about this. especially those involved in other API interfaces. Please let them know about this, NOW.

[anthony]

I have now submitted my modification to "convert" that will allow 'extra compose arguments' to be specified, for the operations -composite and -layers composite. Only these two "convert" operators will make use of extra compose arguments.

The arguments are supplied as a 'comma' addition to the -compose setting, and consists of a 'geometry' style expression, though that style also permits the use of a 'comma separated floats'.

For example -compose dissolve,50 will dissolve the 'source' image by 50% before overlaying. While -compose blend,40,50 will add 40% of the source image and 50% of the destination image. Note that this will make the resulting image 10% transparent if the alpha channel is present in the input images.

The 'displace' compose method is also now available in the same way.

This change is solely in "convert" and does not effect any other API, in any way. However it is a 'hack'. If these special compose methods should also be made available in other 'compose' situations, such as -flatten, -border, -frame, etc... Then a more format Library change should be made for these extra composition arguments, and that will require changes to API's making use of the library.

Part of this update was a update to the CompositeImageChannel() library documentation, detailing the extra 'out-of-band' arguments that the function understands. Developers should check this documentation, if they also want to pass these special composition arguments.


I have yet to update any Command Line API documentation or IM Examples, at this point, though I am sure that will be done in the near future. I do plan to start making heavy use of these special methods in various places. Especially 'displace' in the 'Distrotion/Displacement Maps' area.

I also plan to implement Fred Weinhaus's suggestion of a absolute 'Distort' composition method, that is in many ways similar to the relative 'Displace' method.

The 'Math' method he also outlines, which in some ways is very similar to the 'Blend' method but with extra 'Multiply' and a Bias (-evaluate add) terms could also now be implemented in "convert".

[magick]

For example -compose dissolve,50 will dissolve the 'source' image by 50% before overlaying.

Our preference is separate options which is the way options worked for quite some time except for some recent additions. Something like -set option:composite:blend 50 -compose dissolve. We think it makes the option less convoluted, leaves intact the existing option syntax, allows for expansion in the future, and makes it easier for API users of CompositeImage().

See http://www.imagemagick.org/script/comma ... php#deskew as an example.

[anthony]

Aftre my studies and now my actual experimenting I agree that that is the better solution, as it also means you can set different options for different images. It also means that the extra compose arguments can also be used in ALL compose situations, not just -composite and -layers composite

However it would mean a change to all the API's using it.

Also while the options are passed in the parameter 'blend' in PerlMagick, I would not use that name for extra arguments. In Perl magick I could add a new argument name and set the artifact from BOTH of them.

I will proceed with changing it to use artifacts today (restoring "mogtage.c" as it was before my changes.

[anthony]

I have now implemented the change, to using ImageArtifacts for passing both Extra Arguments ("compose:args") and a change to the use of "modify-outside-overlay" to "compose:outside-overlay".

The ComposeImage() documentation also has been updated. as well as API's for "convert", "composite", PerlMagick, and "Magick Scripting". Other API's will also need to be fixed to follow the new methodology.