Seeking advice on a complex image cataloging project

[jamtat]

I'm a linux user (Arch) who has done a bit of tinkering with image files using imagemagick over the years. So, though I'm not an IM power user by any means, I'm not a complete novice, either.

I've now been called on to assist an elderly artist who is in the early stages of Alzheimers to catalog his work. For the past 10-15 years this artist has worked almost exclusively on a computer and has made extensive use of Photoshop in producing his visual art. Probably 75% of the corpus has been written to CD's, and it looks like he was taking a bit more of an organized approach toward the beginning of that period. But during the last 5-7 years organizational attempts have tapered off, and it is unclear what material is on which CD. I am currently tackling this most recent set of CD's, trying to determine what on them is artwork worthy of cataloging.

My first task has been to copy the content of those more poorly organized CD's to an external hard drive so they'll be more readily accessible. I'm about halfway through the copying process, so I am experimenting with ways to process the data that's now been transferred and to make it more "perusable" so that we can better distinguish which of the data belongs in the catalog. There are a number of digital formats involved, mainly tif, jpg, psd, with maybe a few gifs and bmps.

I've experimented a bit with creating contact sheets, thinking this might be a good way to generate a visual record of images contained on those CD's and thus allowing us to discern the artwork from other types of images (from, say, personal photos or photos that might have been intended as elements in some artwork, but which were never used). Having now looked over more carefully the montage documentation at http://www.imagemagick.org/Usage/montage/#html I'm beginning to think an alternate approach, i.e., visual index images with clickable links (such as you get from running convert 'vid:../img_photos/*_orig.*' vid_index.html) might be a good initial approach to take in helping us discern which images are artwork worthy of cataloging. So my initial questions will be relating to that solution.

A test I ran using that convert 'vid:../img_photos/*_orig.*' vid_index.html command produced what seem to me satisfactory results, with a few caveats. First, some of the file names, which can be fairly long, tend to get a little jumbled as their width exceeds the width of the thumbnails produced and as they obtrude a bit into the adjacent image's caption: is there some way to cause these file names better to break across lines when they are long like this? We may be able to contend with this jumbling since this is really just a preliminary phase in the cataloging process, but if there is some simple way to rectify it, that would help this phase go a bit more smoothly.

Next, in relation to the visual index images approach, the following. It seems to me the varying range of image formats with which we're dealing should be no problem with respect to generating the montage--does that sound correct? However, when it comes to the fuller-resolution images to which the clickable links point, those will likely need to be converted to something typical web browsers can display, correct? So, though I might be able to produce a montage from tif's, jpg's, and even psd's, the browser will be unable to display some of those images (e.g., the tif's and psd's), once a relevant segment of the montage has been clicked, correct?

If I'm understanding that aspect of how visual index images works, then I'll be needing to convert all images to something like png or jpg in order to use that solution. That introduces a new layer of complexity to this task, since I'll need to determine a way of trawling all sub-directories on the external hard drive to which I'm copying, for relevant image files, then converting, very likely reducing them in size, then copying them into a distinct sub-directory, before generating the montage. If you'll bear with me, I'll pose inquiries later about that process.

With this note, I'd just like to get feedback as to whether it seems I'm taking a sound approach at the outset of this project, whether visual index images seems a good solution to implement at this initial phase and, finally, whether I've correctly understood how visual index images works. Your input will be appreciated.

[fmw42]

montage can take any image format and output to jpg or png for display. But you will need to process that image again to put hot spots on it to other full resolution images. They will need to be jpg or png for browsers to display. There are likely tools that can do that and if the nontages are all the same size, then perhaps one set of coordinates can be used for all and just change the names. I have not done this hot spotting for a long time so cannot recommend other software for this. I am not sure if montage will adjust the font size automatically as with label:. But you could rename the files in your OS to some max length. Or you can break the name into two parts with a line break as in one of the examples on the second link below
see
http://www.imagemagick.org/script/montage.php
http://www.imagemagick.org/Usage/montage/

mogrify can process a single directory (no subdirectories) of images with a single command and convert to jpg or png for display and resize along the way and compress if needed. see http://www.imagemagick.org/Usage/basics/#mogrify Otherwise, you will need to loop over all the directories conotaining images and use convert to process them one at a time.

There are many html editors that can add the hot spots.

There are also many platform specific tools for creating catalogs, such as iPhoto on the Mac.

[snibgo]

Are you familiar with Windows Explorer? (I don't mean Internet Explorer.) It's a GUI that shows thumbnails of files in a directory, allows selection of size, with easy navigation between directories and devices etc. Clicking on an image takes me to the appropriate application for the file. I can easily create a directory of symbolic links to files of interest. Doubtless your variety of Linux has something similar.

Web pages are useful for presenting to clients etc, partly because everyone knows how to drive browsers, and I roll my own pages so can include whatever metadata I want. I've never used vid: because (as far as I know) it doesn't give me enough flexibility. Web pages need a bit of thought for thumbnails, layout etc.

There are also commercial systems like Lightroom for organising and catalogueing images.

In my view, a catalogue should have one easy simple linear structure, like a book has. The date of production is one obvious choice for this. Then, of course, you can add whatever other structures/indexes you want: all the green pictures, or photos vs computer art, or keyword access, or the development stages of major works, and so on.

The linear structure is like a backbone for all the other access methods.

[jamtat]

Thanks for your reply, fmw42: you've clarified the matter of the need to convert to web-browser compatible formats, which I suspected would be required for this solution to work. My experimentation with the visual index images script I referenced indicated that it automates the process of inserting/creating links; I just ran that script in a directory full of images and it generated a clickable montage (png I believe) and an index.html. If open that index.html in a browser, it shows me the montage and clicking on any image in the montage causes the browser to open the corresponding fuller-format file. So that command automates the whole process, so far as I can tell. From there, I presume it would be a simple matter of copying over the directory containing index.html and all the fuller-resolution images over to some web server. That's how it seemed, in my initial testing, it should work, anyway.

Just to clarify, I am not yet at the stage of creating a catalog. This is a preliminary phase where I'm just trying to determine what, from among all these files, might actually belong in a catalog. So at this stage I'm not focusing much on how polished the results will look. Once we determine what actually needs to be cataloged, I'll probably create a spreadsheet as the next step. Subsequent to that, perhaps a real, more polished catalog will be produced. But I'm not sure whether funding will persist for that latter phase--we'll see.

[jamtat]

I haven't used Windows with any regularity for some years, snibgo, but I am familiar with Windows Explorer from those years in which I did.And it's true that file browsers, both under Windows and under linux, do typically offer thumbnail views. I should clarify at this point that the reason I'm leaning toward a web presentation is that I am working with two other people on this project (the artist and another helper he's hired), and we all need to be looking at the material in order to sift through it. Since we do not all live and work together on a regular basis (the helper and I only work with him one day per week), a web presentation is desirable for this initial stage of the process. I hope that helps to clarify why I am leaning toward the visual index images solution I described above. I should also mention that this is a low-budget operation, so commercial, proprietary software probably lies beyond the budgetary constraints. Plus, as I mentioned above, this is a preliminary phase of attempting to determine the scope of images needing to be cataloged, and a more polished presentation--if there will be funding for that--can occur only after the scope of the project's content has been determined. Thanks for your further input on this--it's much appreciated.

[Bonzo]

If you are going the web route I would use php with html. You can create a list of thumbnails with filenames and links to larger images with a few lines of code; I am not sure how to do a recursive.

This is a piece of code to find and display the jpg and png files in a folder called canvas and can be expanded to include links to larger images captions etc.:

Code: Select all

<?php
// Directory to read images from
$dir = "canvas/";

// Read the directory and sellect jpg and png files only
$filenames = glob("$dir{*.jpg,*.JPG,*.png,*.PNG,}", GLOB_BRACE);;

// Start the loop to display the images in a table
foreach ( $filenames as $filename ) {

// Get the photo name for use in alt and text	
$image = explode( '/', $filename );

// Remove the extension
$image = substr( $image['1'], 0, strrpos( $image['1'], '.' ));

// Remove the / from the directory path
$dir = str_replace( "/", "", $dir );

// Get the image size for use in the image call
$size = getimagesize( $filename );

// Change _ to white space
$text = str_replace( "_", " ", $image);

// Display the images, use the image name without the extension for the Alt
echo "<img src=\"".$filename."\" ".$size[3]." alt=\"".$text."\" title=\"".$text."\" />\n";
	}
	
?>

[Bonzo]

I should say that the above code is working on a full sized image in the folder. When I used to use this code I would have a folder of thumbnails with the same name as the full sized photo and use the thumbnails in the index. When the thumbnail was selected it opened the large image.

[jamtat]

Thanks for your input, Bonzo. I don't know much about php but am willing to consider various possibilities. I'm curious at the outset as to why you see the solution you've proposed as being superior to the visual index image solution that I discovered (see http://www.imagemagick.org/Usage/montage/#vid ). To my thinking the visual index image method has the advantage of being pretty automated: using that, all I have to do is find and resize potentially-relevant images, copying them into a discrete directory. Then I cd to that directory and run the command

Code: Select all

convert  'vid:../img_photos/*_orig.*' vid_index.html

That creates a montage of all images in that directory, and an index.html that maps each montage region to the corresponding fuller-format file residing in that same directory. At least that's how it worked in my test; I ran the command, opened my browser and pointed it to the index.html file residing in said directory, and it showed me a clickable montage, replete with file names. I was able to look at all the fuller-format images in the directory by simply clicking on the relevant region in the montage. That seems to me a much more automated process than the php solution you've proposed. But maybe, owing to my poor grasp of php, I'm not understanding very well your proposal.

[Bonzo]

It is a similar output to the montage method but the output is more customisable as you can use any html code, css styling organise the photos in different ways. You could also make it show a certain amount of photos per page.
You could have it automated so that when you visited a page it displayed the images in that folder or you could have an index page with links and when you selected the link it could pick certain folders and display the contents.

You can try it if you have access to a server. Change this line from $dir = "canvas/"; to $dir = "./"; and save the program as index.php into a folder with images. Navigate to the folder and you will see all the images in the folder.

I have just noticed an error in my code; replace this line $filenames = glob("$dir{*.jpg,*.JPG,*.png,*.PNG,}", GLOB_BRACE);; with this $filenames = glob("$dir{*.jpg,*.JPG,*.png,*.PNG}", GLOB_BRACE);
If you are unsure what to do the code will still run but there will be an error at the end.

[jamtat]

Thanks for the clarification, Bonzo. So the images get dynamically resized to fit into the table? So that, in the end, you just have a directory of images on the server--and those images can be any size--plus the index.php file. That index.php file then dynamically does the work of creating a sort of thumbnails or montage page, and of linking from that to the fuller-resolution images--is that correct?

[jamtat]

Ok, I've tried Bonzo's proposal and it's an interesting option. I was under a mistaken impression that it generates thumbnails that you click in order to view fuller images. But it appears not to do that, instead simply displaying images located in that directory at their native resolution. I already need to resize these images in any case, so I could certainly just specify a size that fits well within a standard browser window. Thinking . . .

[Bonzo]

Code: Select all

Thanks for the clarification, Bonzo. So the images get dynamically resized to fit into the table? So that, in the end, you just have a directory of images on the server--and those images can be any size--plus the index.php file. That index.php file then dynamically does the work of creating a sort of thumbnails or montage page, and of linking from that to the fuller-resolution images--is that correct?

Yes that is correct.

[jamtat]

I am now--having resized and converted a few hundred images--at the phase of this process where I'm ready to create a sort of web album of the images. Bonzo's approach look attractive in some ways, but I'm not sure exactly how it works.

I did modify the index.php file and placed it within a directory of a few test images, uploading it to a web server. Navigating to the index.php file with my browser, I was able to view all those images. But there was no page of thumbnails which, when one or other was clicked on, would lead to the full-sized images. Rather, the full-sized images simply appeared in a long, unbroken sequence, once I'd accessed index.php.

Looking at Bonzo's test page link--where there are clickable thumbnails--there seems to be some additional html code on the page where you initially land. So, where does that code come from? Is it supposed to be auto-generated? Am I supposed to code it myself? It seems to me something's missing--quite probably in my understanding.

In any case, if I can't figure out better how that page of clickable thumbnails is to be generated, I would probably go with the visual index image solution I found and which, in my initial tests, seemed to work suitably enough.