Defaults or how to find MAGICK_TEMPORARY_PATH

[spaceguns]

Context - can skip to ISSUE if you want to get right to what I am trying to do.
Apologies for a possible newbie question, I am a hobbyist. I am working on an image program at home and part of it involves processing pdf files. Currently all this is written in python and running on a windows machine and utilizing Wand/ImageMagick/Ghostscript to do the pdf processing. Occasionally, a CorruptImageError is thrown, which I can live with/analysis of is for a different issue.

The problem is when this happens and ImageMagick chokes it leaves behind some often sizable temp files magick-(numbers/letters) at MAGICK_TEMPORARY_PATH. In my case this is at %LOCALAPPDATA%\Temp

I have researched this and it seems to be an acknowledged issue with the solution being to run an occasional script/cron job to clear those out. What I would like to do is add this to the end of my program so once the program completes it will go through the MAGICK_TEMPORARY_PATH location and clear out all the magick- files. I can do this at %LOCALAPPDATA%\Temp but I want this to be able to run anywhere/on any os/settings path.

ISSUE
How would I best locate MAGICK_TEMPORARY_PATH in an os/install independent way?

In python I can call sys.platform.startswith(osname) to identify the os I am on but even then assuming MAGICK_TEMPORARY_PATH will be the install default for that os wouldn't be the best way to go.

Is there a way, preferably in python, that I can get the MAGICK_TEMPORARY_PATH so I can then run through it?

Alternatively, and this isn't the ideal solution but works for beta, can someone list the defaults for MAGICK_TEMPORARY_PATH?
Windows: %LOCALAPPDATA%\Temp
Cygwin: ???
Linux: ???
Mac: ???

[fmw42]

If in Unix and you put MAGICK_TEMPORARY_PATH in your .profile or .bash_profile, then you can cat your profile file and parse for MAGICK_TEMPORARY_PATH. I do not know windows, so cannot help on that end.

[spaceguns]

Thanks for the quick reply and I will look into that.

I may be able to code where it checks .profile and .bash for the presence of the setting, but if it is not found can fallback to the suspected default.

Do you know what the default temp location would be for Unix systems?

Thinking of other possible leads to solving this is anyone is aware of an shell/terminal command which may return the MAGICK_TEMPORARY_PATH? I can probably have one of those run then capture and parse the output.

[spaceguns]

Update: Using that train of thought I can possibly get it in python if it was custom set using
im_temp_path = os.environ['MAGICK_TEMPORARY_PATH']

However, I can't prove that on my system since it is appears that it is falling back to os.environ['TEMP'] since I don't have MAGICK_TEMPORARY_PATH customized.

This may provide a solution, maybe. If a few other users can check results and let me know what the non-windows environ falls back to I should be able to check for MAGICK_TEMPORARY_PATH and if it doesn't exist go to the default fallback.

[snibgo]

If you have set MAGICK_TEMPORARY_PATH, then that immediately solves your problem, doesn't it? The problem is when that hasn't been set.

I think IM gets the temporary path from function GetPathTemplate(path) in resource.c. That source code gives the environment values that are searched for, on diffferent platforms. Note that policy.xml might define the temporary location, and that will override all other settings.

EDIT: to find the path used from a command line, use a delegate that uses %u, with "-verbose", and parse the text output. For example:

Code: Select all

f:\web\im>%IM%convert -verbose anypdf.pdf NULL:

[ghostscript library 9.19] -q -dQUIET -dSAFER -dBATCH -dNOPAUSE -dNOPROMPT -dMax
Bitmap=500000000 -dAlignToPixels=0 -dGridFitTT=2 "-sDEVICE=pngalpha" -dTextAlpha
Bits=4 -dGraphicsAlphaBits=4 "-r72x72"  "-sOutputFile=C:/Users/Alan/AppData/Loca
l/Temp/magick-9716IjIQZTenPOQ9%d" "-fC:/Users/Alan/AppData/Local/Temp/magick-971
6SDTYWpT_bDjC" "-fC:/Users/Alan/AppData/Local/Temp/magick-9716nEGea0bRxec3"C:/Us
ers/Alan/AppData/Local/Temp/magick-9716IjIQZTenPOQ91 PNG 200x100 200x100+0+0 8-b
it sRGB 419B 0.016u 0:00.016

From this, we see IM is using C:/Users/Alan/AppData/Local/Temp/".

A custom dummy entry for delegates.xml could probably be written that would be easier to parse.

[spaceguns]

Thanks snibgo. That is a great lead and I can probably grab it using regex

Working solution for Windows. Would love input on how to maybe do this better and if anyone else can test, particularly on other OS versions.

Python Code - (for any newer folks following you need to have this mapped or be working in the dir with convert)

Code: Select all

import re
import subprocess
startupinfo = subprocess.STARTUPINFO()  # Suppress terminal popup
startupinfo.dwFlags |= subprocess.STARTF_USESHOWWINDOW  # Suppress terminal popup
IM_output = subprocess.check_output('convert -verbose test.pdf NULL:', startupinfo=startupinfo)
temp_loc = re.search('(?<=-sOutputFile=).*?(?=/magick-)', str(IM_output)).group(0)
print(temp_loc)

Thinking of it adding this to the front end of the program as a "is this even installed correctly, if not drop processing of postscript files" would be a good check using a test.pdf included with the code. Would make sure everything will run and allow me to grab the temp directory for cleanup at the end.

[snibgo]

... using a test.pdf included with the code.

Or create a PDF using IM:

Code: Select all

convert rose: test.pdf

IM doesn't use delegates for that operation.

[spaceguns]

... using a test.pdf included with the code.

Or create a PDF using IM:

Code: Select all

convert rose: test.pdf

IM doesn't use delegates for that operation.

Good idea and will try it out. Right now I am on a hunt for exactly what is causing this issue overall before I go with a "nuke everything magick-" solution at the end.

Once I am happy with a solution I will try to follow up with the Python code I am using here and with the Wand folks to see if it can help out others.

[dlemstra]

Could you open a bug report when you have something that is reproducible with the latest version of ImageMagick @spaceguns?

[spaceguns]

Could you open a bug report when you have something that is reproducible with the latest version of ImageMagick @spaceguns?

Will do! Hobby project so it may be a while but once I hammer out a solution I will also run the same tests on the most recent version and post results and any workarounds.

At this point I may look at ditching Wand and just coming up with my own solution direct with ImageMagick which would further isolate any issues.

[spaceguns]

In my efforts to just move away from Wand and implement direct ImageMagick commands and clean any errant temp files in a system agnostic way could a few others review/test this/let me know if it looks horrible?

Python 3.6 code with ImageMagick 7.8.0-14 installed. Hefty comments and the critical os.unlink line commented out to keep someone new from tinkering and accidentally nuking a folder they didn't mean to. Didn't bother with graceful error handling just yet and currently set to print results rather than delete the files. I am hoping this will work cross system and independent of MAGICK_TEMPORARY_PATH being set.

Code: Select all

# Setup and prep using ImageMagick 7.8.0-14
import os
import re
import sys
import subprocess

startupinfo = None
# Windows suppress terminal popup
if sys.platform.startswith('win'):
    startupinfo = subprocess.STARTUPINFO()
    startupinfo.dwFlags |= subprocess.STARTF_USESHOWWINDOW

# Step 1 - test functionality by making the pdf
subprocess.call('magick rose: im_test.pdf', startupinfo=startupinfo, shell=True)
# Step 2 - generate the verbose output which seems to trigger on converting an existing image
IM_output = subprocess.check_output('magick -verbose im_test.pdf NULL:', startupinfo=startupinfo)
# Step 3 - kill that test file
os.unlink('im_test.pdf')
# Step 4 - regex the temp directory
im_temp_loc = re.search('(?<=-sOutputFile=).*?(?=/magick-)', str(IM_output)).group(0)

# Step 5 - Run rest of program
# All those calls to your actual program stuff goes here.
# May involve some temp files getting generated and failing to delete when ImageMagick fails.


# Step 6 - Delete magick- files from temp directory
# Define function to delete temp files based on path and file prefix
def clear_temp_files(temp_folder_path, file_prefix):
    temp_file_count = 0
    for filename in os.listdir(temp_folder_path):
        if filename.startswith(file_prefix):
            # DON'T UNCOMMENT OS.UNLINK BELOW UNTIL YOU ARE SOLID ON RESULTS! WILL DELETE THINGS!
            # os.unlink(os.path.join(temp_folder_path, filename))
            # COMMENT below print and UNCOMMENT above os.unlink after testing to delete files.
            print(os.path.join(temp_folder_path, filename))
            temp_file_count += 1
    return temp_file_count


# Execute and get file count
deleted_temp_files = clear_temp_files(im_temp_loc, 'magick-')
print('Temp files deleted: ' + str(deleted_temp_files))

Good/bad opinions appreciated. I don't know what I don't know so I would really appreciate pointers particularly if this just doesn't work on a non-windows system.

If/once I move away from Wand and directly to IM for this I will try to better isolate the errors/what's actually causing the failures and jam out some bug reports.