speed start up by moving config files to ramdisk?

[NicolasRobidoux]

Since ImageMagick searches for config files at start up, can one speed up start up by putting config files so that they are found quickly?

To the extent of putting copies in ramdisk, and setting MAGICK_CONFIGURE_PATH so it points there? Or something like that?

(Of course this will become somewhat moot once IM can be run in daemon mode.)

[anthony]

Note that Im does not load configs until they are actually needed!

As such if you don't use a color name, the colorname config is not loaded.

Mind you that may have changed as it looks like most color names are not built into ImageMagick!
But for other configs that should still be the case.


Where the config is on the path does not matter that much as typically the config is looked for (and loaded) from every directory if present. for the actual lookup the first match is the one that is actually used, as such a font declared in a system file will be delected over a font of the same name from a user file. That is user (or later) config file can not override system configs.

This is a security requirement, especially when it comes to configs involving executables or coder modules.

As such the user can not for example have a personal replacment for "show:" delegate, but can still create his own "myshow:" delegate. (I have - for experimentation with a faster 'spawning' technique).


Now how would you invisage speeding this up?

[NicolasRobidoux]

First, the context:

Monster machine that only resizes, frames, mosaic and changes the format of images (everything ends up as .jpg or png, with some intermediate images stored as .ppm to feed directly to cjpeg and relatives, and some in .mpc to speed up computations involving multiple operations on the same input image). Eventually it may also do some colour correction. That's all it does. So, I can customize ImageMagick all I want, and the person who installs stuff is definitely root. I don't care if I mess things up for other users or uses, because there are none. IM is compiled from source.

I prefer not writing a monster ImageMagick script that "does everything." Maybe I should. But there is a certain cleanness to keeping things separate. And then PERL/*nix can drive the concurrent processing of independent tasks instead of IM. (I'm certainly less enclined to write a monster ImageMagick script knowing daemon mode is coming down the pipeline, although the client runs a lagging version of IM/OS and this won't change soon.)

Idea:

When the specific convert commands we use start up, they look for config files (I assume). (I've not taken the time to figure out which ones.)

To speed up the "look up", put the config files (empty ones if there is nothing to configure) somewhere where they will be found and loaded fast. This means ramdisk, set up so that it is at the top of places where the config files are searched.

These kinds of configure files: http://www.imagemagick.org/script/resources.php.

That's it!

I've not tried anything. This post is putting out "feelers".

[magick]

The configure script supports --enable-zero-configuration to enable a contained, embeddable, zero-configuration ImageMagick. All configuration files are loaded from memory.

[NicolasRobidoux]

The configure script supports --enable-zero-configuration to enable a contained, embeddable, zero-configuration ImageMagick. All configuration files are loaded from memory.

Very cool!

Thank you.

[NicolasRobidoux]

Looks like I'm definitely not the first one who wanted "don't look for config files": http://studio.imagemagick.org/pipermail ... 15536.html.

[NicolasRobidoux]

Is there documentation about this option anywhere? (Besides reading IM's configure scripts.)

[magick]

Nope. Its a one-off, seldom used, so we haven't documented it yet. The --enable-zero-configuration option defines MAGICKCORE_ZERO_CONFIGURATION_SUPPORT. That define triggers conditional compilation such that the delegates, locale, and font configuration files just use the built-ins (in memory). It also forces the embedded memory manager, rather than malloc(). If you have any problems or suggestions for improving this option, let us know.

[NicolasRobidoux]

Important According to http://www.imagemagick.org/script/changelog.php, the ./configure flag was called --enable-embeddable up to IM 6.7.2-5 inclusive, after which it became --enable-zero-configuration.

[NicolasRobidoux]

Compiling with this flag certainly does not make a huge difference in total runtime.

I've only checked putting a rounded frame on a 600x600 .mpc file and converting to .jpg. (The job discussed here: viewtopic.php?f=1&t=20531)

[magick]

Right. Loading the configuration files is lightweight and most are not loaded unless you need them (e.g. named colors are only used if you need a named color: red, cyan, etc.).

[anthony]

It also forces the embedded memory manager, rather than malloc().

How does that work? malloc is the way to get memory from the system.

What do you use instead of malloc?

Why was this needed for 'zero configuration files'?

[magick]

We use sbrk().

We could break out the embedded memory system into a separate option, we have simply not gotten around to it yet.

[anthony]

Oh. I seen a direct kernel API call (man section 2) for large data segments of memory, rather than C library calls.

Thanks for this info.

I did notice previous IM has code for doing its own memory allocation using the results of a research paper.
"Fast Memory Allocation using Lazy Fits." written by Yoo C. Chung.

[NicolasRobidoux]

...
We could break out the embedded memory system into a separate option, we have simply not gotten around to it yet.

(Just a) suggestion: --enable-zero-configuration only does what it says, and then --something-something-memory-something or even --enable-embeddable, which would automatically set --enable-zero-configuration.

(And yes, I understand long to do lists. I'm not expecting anything here.)