I/O: plain filenames, streaming with callbacks

[eevee]

I'm building an image library for Python on top of MagickCore, and for the life of me I can't get around these two issues with reading/writing files.


1. The `convert` utility has its own microsyntax around filenames, for specifying the format, extracting a specific frame, cropping and resizing, etc. This is mighty convenient when dealing with the terse environment of a command line. Unfortunately it seems that some of this behavior (the type prefixes and frame extraction, at least) exists deep in the core code, even within ReadImage.

Is there any way to circumvent or disable the special syntax? A high-level language API doesn't need the same kind of shorthand, and I don't want end users having to worry about filenames not actually being filenames in some cases, especially when it's not well documented what parts of the filename syntax ReadImage supports.

I tried adding backslashes before colons and brackets, but GetPathComponent doesn't seem to respect backslashes. My current solution is to fopen the file myself and only pass ReadImage a FILE*, but then ImageMagick can't fall back to guessing the filetype based on the extension.


2. Python has an API that allows an object pretend to be a filehandle. Such an object can then stream from a database, generate data as it's read, etc. I'd like to be able to read images from and write images to such objects, and it seems like this should be possible; most decoders already use the ReadBlob abstraction to read chunks at a time from an opaque source.

I can't bridge this gap, though. Virtually all of stream.c is only defined in stream-private.h, making it off-limits. There's a FifoStream blob type that almost seems appropriate, but ReadBlob doesn't know how to read from such blobs. Right now the best I can do is create an in-memory buffer of the entire encoded image, then pass that from Python to BlobToImage or vice versa.

Is there a way to read/write based on arbitrary callbacks?

[magick]

Image filenames are overloaded and baked into the ReadImage() method. As you suggested, the only way around it is to provide your own stream in the ImageInfo structure (e.g. image_info->file, image_info->blob, or image_info->stream).

[anthony]

I have been thinking about 'exact filename' type image reads. No shell expansion, no '[...]' syntax. just a optional coder(s) and filename.
However this is right near the bottom of my todo list.

However you can read/write images from pipelines ("-"), or some other pre-opens file descriptor ("fd:N")

For example here is one such test.

Code: Select all

  convert rose: rose.png    # just save an image to disk as a source
  exec 5<rose.png            # have the shell open this file on file descriptor 5

now for the fun bit...

Code: Select all

  convert fd:5   info:
fd:5 PNG 70x46 70x46+0+0 8-bit DirectClass 7.01KB 0.000u 0:00.019

and clean up

Code: Select all

  exec 5<&-        # close file descriptor
  rm rose.png

Of course that is just an example. You can open pipelines in the API, or use named pipes too.

This type of thing is at the heart of the IMv7 development, which can even read the options it is to process from files or pipelines, not just images.
See IMv7 Scripting Development Notes
http://www.imagemagick.org/Usage/bugs/I ... ipting.txt

WARNING: IMv6 currently reads all images from a pipeline it is given. At this time you can NOT ask it to only read one image (using a well behaved streaming image file format). Even if you ask it to just read the first image it will still read the whole file stream! The Ability to read one image, and the ability to disable some or all of the filename syntax, is something that I want to appear in IMv7.