Page 1 of 1

IM 7 Improvements and Scripting

Posted: 2016-12-18T16:18:37-07:00
by fmw42
I have split this off from where it was posted, since it was getting off-topic.

__________________________________________________________________


In IM 7, grayscale images have only one channel and not 3 equal channels, so processing would be faster. Also you can do % escapes and %[fx:...] calculations in-line. IM 7 is less forgiving than IM 6 to improper syntax. There are a few changes you must know about. See http://imagemagick.org/script/porting.php#cli

IM 7 Improvements and Scripting

Posted: 2016-12-18T19:42:51-07:00
by anthony
Other changes.... It also allows 'magick' scripts (same result regradless of UNIX/Dos), complex sub-scripts can be called as 'functions', and pipelined scripts into commands (so a program can control a 'magick' sub-process on the fly).

This is why IMv7 option handling was re-organised, allowing it to do strict sequential processing of options, as it reads them from files or pipelines.

IM 7 Improvements and Scripting

Posted: 2016-12-18T21:32:18-07:00
by snibgo
Anthony wrote:complex sub-scripts can be called as 'functions'
That could be useful. Is it documented somewhere?

IM 7 Improvements and Scripting

Posted: 2016-12-18T22:15:38-07:00
by fmw42
anthony wrote:Other changes.... It also allows 'magick' scripts (same result regradless of UNIX/Dos), complex sub-scripts can be called as 'functions', and pipelined scripts into commands (so a program can control a 'magick' sub-process on the fly).

This is why IMv7 option handling was re-organised, allowing it to do strict sequential processing of options, as it reads them from files or pipelines.
Anthony, would you provide documentation and examples of magick scripts, please. No one but you have any idea how they work or what they can do. The limited documentation from your notes is hard to understand (at least for me). I think a tutorial is really needed to get any benefit from such scripts.

IM 7 Improvements and Scripting

Posted: 2016-12-19T19:40:36-07:00
by anthony
Magick scripts are basically just the command line options as a script.
For example in the ImageMagick sources is a sub-directory "api_examples".

Here is a 'normal' shell script

Code: Select all

#!/bin/sh
#
# Assumes the "magick" command has been installed
#
magick -size 100x100 xc:red \
       \( rose: -rotate -90 \) \
       +append   show:
this works much like you would probably expect.
Here is a Magick script

Code: Select all

#!/bin/env magick-script
#  script.mgk
# Assumes the "magick-script" symlink to "magick" command has been installed
#
-size 100x100 xc:red
( rose: -rotate -90 )   # you can have comments in script too!
+append   -write show:
It is basically exactly as per the command line options but without needing to backslash end of lines, parentheses, or other shell meta characters like !
but you can include comments in the middle of the script!

It can be run using

Code: Select all

   magick -script "script.mgk"
Or using the comamnd

Code: Select all

   magick-script "script.mgk"
Or if you used the 'HASH-BANG' scripting "#!/bin/env magick-script" (as showned above) which runs that file as a "magick-script" as it is found on your normal command path.

You should even be able to make DOS executable magick-scripts (as I made a '@' at the start of a line equivelent to a comment for DOS scripting, same for ':' or '#' for UNIX scripting)

Code: Select all

@echo This line is DOS executed but ignored by Magick
@magick -script %~dpnx0 %*
@echo This line is processed after the Magick script is finished
@GOTO :EOF
#
# The rest of the file is magick script
-read label:"This is a Magick Script!"
-write show: -exit

The scripts was supposed to also be able to read/process other image arguments, (using -args option). More imprtantally it was ment to allow you to implement 'montage-like' scripts. That is the script processes all (but the final write) like montage does, the script them can process the images in some way, and write the result to the file argument. I hoped to eventually replace montage itself with just a script and do away with the 'mess' of that commands code!

But I did not get to implementing that as I wanted to get -fx handling settings -- which is where I burned out. :-( So lets leave the magick-script at that for now.

The "function" type aspect, now that I am looking back at it, was also NOT completed.
The key here was the "-return" option.

Now suppose you create a 'function' magick script, that assumes you have an image in memory when it is called.
Say the file is called "half.mgk" and just resizes the image by 50%, but it could to just about any complex operation.

Code: Select all

# magick-script which expects a image already in memory
-resize 50%
-return
Now you can run it like...

Code: Select all

magick rose: -script half.mgk show:
You read in your images, run them through your 'function script' which 'returns' to let you continue processing.


The goals (and unimplented options) are documented in...
http://www.imagemagick.org/Usage/bugs/I ... ipting.txt

But the biggest goal -- pipelining commands into magick -- is working!
Even if 'co-processing' using a magick sub-process isn't quite working (I just tried using it for co-processing)
(Co-Processing: http://www.ict.griffith.edu.au/anthony/ ... _hints.txt

Seems like this area is needing work. and I don't have the time to get into it! :-(

Re: IM 7 Improvements and Scripting

Posted: 2016-12-20T00:29:55-07:00
by fmw42
So what is the benefit of using

Code: Select all

magick rose: -script half.mgk show:
vs

just making a string for the commands and doing

Code: Select all

string="-resize 50%"
magick rose: $string show:

Can you provide a better example that shows the benefit of the magick script?

Re: IM 7 Improvements and Scripting

Posted: 2016-12-20T22:02:12-07:00
by anthony
The benefit is you can create complex functions... for example

Code: Select all

magick image.jpg \
            -script circle_mask.mgk \
            -script add_border.mgk \
            -script 3d_warp.mgk \
            -script thumbnail.mgk \
            output.png
You can call a -script multiple times if needed. It may be we can get true functions (code blocks) at some point!

Also using magick scripts will work the same on UNIX, Mac, DOS. And does not care about end-of-line issues between them.

The scripts do not have problems with the need to escape things like percent, exclamation marks, percents, unicode strings, end of line escaping, etc. etc. etc. that is required by the wrapping shells (bash-csh-tcsh), DOS scripting, or for calls from PHP, Perl, Ruby, Python, etc. The same script will be read and work in all the environments.

And you can add comments in the script, making them easier to understand what various sections is doing. Commenting also means you can add/remove code in the script during development, by just commenting/uncommenting out various operations, where with the command line you would need to move the lines completely out of the command. It also opens the possibility for true functions where option sequences are 'stored' and then 'called' and that opens up possibilities of if-then-else conditionals, and even code loops.


The current problem is only the key basics to read options from a file or pipeline, instead of just the command line array, are present. It is the not finished. (See what I envisage in IMv7_Scripting.txt)

For example allowing the script to read extra arguments like images, strings or numbers from the command line, or simply process all the arguments as 'normal' then do extra processing (such as montage does) before writing to the final output image. At the moment arguments to scripts will need to be given in 'settings' defined before the script is called.

EG: you currently need to do...

Code: Select all

magick  image.png -set label 'Ants Minecraft Video' -define arg:number=023  \
              -script video_thumbnail.mgk  \
             title.png
instead of having a autononious script that can read arguments like...

Code: Select all

video_thumbnail.mgk  image.png 'Ants Minecraft Video' 023  title.png
I used that as an example because at the moment I have a shell script generating such images....
Image

Re: IM 7 Improvements and Scripting

Posted: 2016-12-20T23:11:59-07:00
by snibgo
I'm looking forward to being able to do that. Currently (v7.0.3-5) we can't. V7 scripts don't return, so we can't call one after the other. (And scripts can't call other scripts.)

Re: IM 7 Improvements and Scripting

Posted: 2016-12-20T23:42:04-07:00
by fmw42
I am trying to follow your comments, but it is hard determining what is currently possible from what you planned. Which of your examples above actually work. For example your video_thumbnail.mgk has not been presented. Does it work? Can you detail it or provide another full example including the script code and example results that works currently and does not require future enhancements.

Perhaps you can outline what features are currently available and if possible provide examples.

Re: IM 7 Improvements and Scripting

Posted: 2016-12-21T17:46:54-07:00
by anthony
At the moment the features I know is working.

Code: Select all

magick -script file.mgk
Where file.mgk contains all the command line options that you would normally put on the command line.
but without all the shell escapes, for shell meta characters and end of lines, and with '#' comments
or full line comments if the line begins with any of '#:@' That later allowing for self launching DOS scripts.
The Primary syntax source is the header of the script tokenizer "MagickWand/script-token.c"
Quote and whitespace escaping is based on what UNIX Bash does.

At the moment the script exits when finished. What it is suposed to happen is that the script continue processing either the calling script of command line arguments (if any). The final 'write' output file argument is optional after a -script option).

The script does not need to be a file but can be standard input.

Code: Select all

magick -script -
You can now type options interactively.... ([what I type, magick outputs)

Code: Select all

magick -script -
2016-10-27_19.48.12.png
-identify
2016-10-27_19.48.12.png PNG 1280x720 1280x720+0+0 8-bit sRGB 1.241MB 0.060u 0:06.459
-exit
Or you can 'pipe' commands from your script into the command. for example like this "mogrify" like script...

Code: Select all

mkdir thumbnails
for image in *.png
do
    echo >&2 "Processing image $image..."
    echo "\"$image\" -resize 100x100 -write \"thumbnails/$image\" -delete 0--1"
done | magick -script -
This is working though I am seeing 'property' errors for some reason!

-------------------------------

This reading from a file stream should allow you to 'co-process' with magick.
That is you run a "magick" command in the background from your shell, C, PHP, Python script, and send it commands to process, while reading back information you request (using -print or -identify options) from the background process.
For example you can have "magick" holding images in memory while the "master" program can ask for information about the images, and request them processed in various ways.

This is actually how most Database and Internet Services actually works! When you make a request from a web server or database, the server processes the request then returns a response, and that gives us a nice way to test magick co-processing capabilities...

For example using "netcat" or "nc" to remotely control a "magick" command on a remote system
Assuming the machine firewall has a open port on 8080 and no other program has a service using that port...
(WARNING this is highly insecure and only an example)

On a Image Server in a image directory...

Code: Select all

nc --listen --sh-exec 'magick -script -' 8080
Now on another machine

Code: Select all

nc image.server 8080
-print "image count: %n\n"
image count: 0
-read "*.png[100x100]"
-print "image count: %n\n" # let us know when read is finished, and report how many images
image count: 101
-delete 0--4 # delete all but last three
-print "image count: %n\n"
image count: 3
-format "filename: %f size: %wx%h\n" -identify
filename: 2016-12-16_19.29.43.png size: 100x56
filename: endrod_fence.png size: 100x56
filename: tree_house_stair.png size: 100x56

+append
-print "image size %wx%h\n"
image size 300x56
-exit
That is a bare bones co-processing example, where you are controlling a remote "magick" program running on a different machine! The comment that is sent to the remote "magick" command is ignored. That print basically provides a way to discover when the previous 'read' (which could be a long time) or other sequence of image processing commands has finished. The setup however is not very 'error friendly' as it stands, if something goes wrong with this setup you do not get any notification of the error. But that can also be fixed in the way the "magick" command is launched.

Re: IM 7 Improvements and Scripting

Posted: 2016-12-21T17:54:26-07:00
by fmw42
So does the script create output images? Or do you have to specify -write to get outputs? Or have an output reference outside the script?

Re: IM 7 Improvements and Scripting

Posted: 2016-12-21T18:04:51-07:00
by anthony
use -write

The only place a -write is implied is on the end of the command line.
-read does remain implied for any option not starting with a '-' or '+', but in a script it may be better of also using -read to be explicit.