Noob Question: batch conversion + transparency

[illyume]

My problem:

I have a large number of files in a somewhat complicated system of folders that I need to convert from .tif format to .png format, while specifying white (#FFFFFF) as fully transparent. We're talking 100+ folders, and 5,000+ files--which is why I need to batch convert, instead of just going through photoshop and doing each one by hand.

It looks like ImageMagick may be the best choice for this task, but I honestly have no idea how to get it to function properly. Can I get some help with this?

[anthony]

You can convert white to transparency.
http://www.imagemagick.org/Usage/color/#opaque

However is that what you really want? Is white the background? is the edges mixed color anti-aliased pixels? Basically are you doing the right thing?

Give use a small example image (post a link to the image)

[illyume]

These are a bunch of older files, some of which were converted to .wmf and various formats at some point, but the .tif files are of the highest quality, and are the ones I'm needing to convert to .png now.

I would obviously get the highest image quality if I went in and hand-adjusted each one. White is the background yes, although I believe there's a chance there may be some pure-white pixels within the images themselves, with a few of the files, and some of the files may have some anti-aliasing around the edges that will need to be cleaned up later.

However, with the enormous volume of files I need to convert and apply this transparency mask to, I would much prefer not to spend the time it would take to do this all piece-by-piece. If there are minor hitches in how the images turn out looking, I can always go in and fix things up later, on the ones that turn out to need adjusting.

Warning, the image file is very large (3 MB)

P.S.: The link you posted was broken. 404 Error, for the lose.

And, I kind of need some help just understanding the basics of how the ImageMagick program works. I'm feeling seriously noob right now. After looking through the program's documentation some more, I'm realizing that the primary method of utilizing this program is from within other programs--which I could do, if necessary, but it would be much more convenient if I'm able to run the functions through windows's command prompt.

Anyway, any help and advice I can get would be beneficial. Thanks!

[anthony]

I fixed the link.

Though the link refers to straight white to transparency replacement, a little further does is using flood fill to replace OUTSIDE white colors only. that is only white colors connected to each other around the image, OR if you add a temporary white border, whites connected to the edge of the image.

However I am not really concerned about the pure-white colors in (or around) the image, but a mix of white and image colors along the edges of the image. The above color replacement methods is a all or nothing replacement. You either replace the color, or you don't. the result is a loss of the anti-aliasing edges, and posibly 'halo' effects.

For more details and better but more complex background conversion see
Background Masking in Channels, Masks, and Transparency
http://www.imagemagick.org/Usage/channe ... k_creation

NOTE: while some examples deal with a known background pattern, you can still use these as you know the background pattern.. solid white.


Gad that image is hugh! you could have submitted a cropped a version so around just one of the flakes!

Cropping a section and taking a close look at the actual pixels forming an edge...

Code: Select all

   convert BDR\ Snow\ Flakes.tif -crop 60x60+105+555 -scale 500%  snowflake_edge.png

Yes it is as I feared, the edge is very blurry, even more than is needed for normal anti-aliasing. Lots of object colors mixed with the white background color. Replacing white with transparency, then overlaying will result in a strong 'halo' effect.

Even the 'black' outlines of the outlined snow-flak is not complete with large gaps in them.

If you plan to resize the images to at least half this size, this is not a big problem, just do the color replacement first, then resize. The Resize will then re-add the appropriate anti-aliasing edge pixels with transparency.

But this will not work well if you do not plan on resizing the final image!

You best bet at this time is to use
Masking with Anti-Aliased Edges
http://www.imagemagick.org/Usage/channe ... antialised

This is not nice, but if the process is placed into a script, you may be able to use fixed 'fuzz' range parameters
to get a reasonable anti-aliased removal of the white background.

[illyume]

Hah, sorry about not cropping the image! That should have been an obvious thing to me!

Anyway, thanks for the help! This is a lot more than I was expecting, and now that I've figured out how to get the imagemagick functions working, (I feel like an idiot for not having figured it out sooner--it's so simple,) I think I'll be able to make the these images turn out looking halfway decent. I'm actually pretty excited about this. Thanks, again!

---

And another few questions, and ones that may be just as noobish as my last one, but if someone has an answer to this, it will help me save a bunch more time, and will be very helpful.

For some reason, I'm having a difficult time getting the convert command to run well for all the files within a folder (with the *.png and *.tif wildcards) -- but the mogrify command/program/whateveryouwouldcallit works fine. I think that's the one I'm supposed to use? Anyway, I've been trying it out, and it looks like this command works properly:

Code: Select all

mogrify -format png -alpha set -transparent white *.tif

Is this correct? Should I be using a command similar to this? To be noted: I haven't tried playing around with the Masking with Anti-Aliased Edges yet; but to be honest, a good number of the images are looking just fine without that. Most have sharp, aliased borders, and go from a pure white background to the image themselves. The sample I put up here was a bit of an exception, actually. Still, I'm planning to work out how to make those few look at least a little smoother, too.

But anyway, my big question:

Is there a fairly simple way to cause this code to run for the folder you're in, as well as all subdirectories, so that I don't have to navigate through this complicated system of directories and run the command in each folder--and instead just run it once?

I'm thinking, if there's nothing simpler, that perhaps creating a .bat file of some sort might work?

P.S.: I've attempted using the /s parameter in the cmd window, and things glitch up when 'convert' tries to do stuff with the folders. I'm not sure what exactly the problem is, unfortunately.

[illyume]

Okay, so, this is the function I have now, in its own .bat file and it works quite nicely, on the couple of images I've tried. I've got it set up right now to be called by another .bat file that simply sets the variable 'file' to whatever the original .tif file's name is that I want to convert.

And Update again:

I have these two .bat files:

BatchConvert.bat

Code: Select all

@ECHO OFF
ECHO BatchConvert Program
PAUSE
for %%f in (*.tif) do call script.bat %%f
PAUSE

which calls
script.bat

Code: Select all

::Convert Function

set file=%1
set file=%file:~0,-4%

convert %file%.tif ( +clone -fx p{0,0} ) -compose Difference -composite -modulate 100,0  +matte -channel B -evaluate set 0 img_difference.png

convert img_difference.png -fill blue -fuzz 2%%  -opaque black img_outm.png
convert img_difference.png -fill blue -fuzz 4%%  -opaque black img_inm.png

convert img_inm.png -channel blue -separate +channel -negate %file%.tif +swap +matte -compose CopyOpacity -composite img_inside.png
convert img_outm.png -negate img_inm.png -channel blue -separate +channel -compose multiply -composite img_aliasing_area.png
convert img_difference.png -channel red -separate +channel img_aliasing_area.png +matte -compose CopyOpacity -composite -background white -compose Over -flatten -normalize  img_maap.png
convert img_maap.png img_aliasing_area.png -compose multiply -composite -negate -background black -channel A  -combine img_edging.png
convert img_inside.png img_edging.png -background none -flatten %file%_final.png

   ::Del
   del img_difference.png
   del img_outm.png
   del img_inm.png
   del img_inside.png
   del img_aliasing_area.png
   del img_maap.png
   del img_edging.png
   :EndDel
echo %file% converted.
:EndConvertFunction

This works marvelously, in a single folder. Now I need to get it to somehow run this same process in multiple subdirectories (2-4 directories deep) of the folder it's in.

[anthony]

The batch script for a single file can be improved my merging the multiple convert commands together so more processing is done in memory without needing to save and restore the intermediate images.

The only reason this is not done in IM examples, is I am trying to explain the whole process.

You may also like to make the low and high fuzz limits of the masking arguments into the script.


As for a wrapper batch script to recursively do every image in sub-directories, I am suprised the IM Examples, Windows Usage guide does not have one. I did not write it (I am a UNIX programmer not windows, sorry).

More than likely it is simply a matter of getting the wrapper script to change directory into any sub-directory it finds, than call itself again in that sub-directory. Perhaps you could mail Wolfgang Hugemann (see bottom of that page) for help, with this. Would make a nice addition to the Windows Guide.

If I was under UNIX, I would use a 'find' command to call the batch script on even TIF file it finds! (see UNIX examples of this in. Mogrify, Batch Processing Alternatives

Actually I think a windows 'find' command is available somewhere.

[illyume]

Okay, I've gotten my question on the subdirectories thing answered elsewhere, and my .bat files are working just as I originally had hoped they would. I don't necessarily need to smoosh up the conversion process into a single command, since it works fine as is (and I have the temporary files auto-delete right after running through each image) and I don't particularly care to mess with figuring out how to compact the code. However, I wouldn't mind understanding how to do it either--so if you feel like being extremely generous, how would I compact the following?

Code: Select all

convert "%file%.tif" ( +clone -fx p{0,0} ) -compose Difference -composite -modulate 100,0  +matte -channel B -evaluate set 0 

img_difference.png
pause

convert img_difference.png -fill blue -fuzz 2%%  -opaque black img_outm.png
convert img_difference.png -fill blue -fuzz 4%%  -opaque black img_inm.png

convert img_inm.png -channel blue -separate +channel -negate "%file%.tif" +swap +matte -compose CopyOpacity -composite 

img_inside.png
convert img_outm.png -negate img_inm.png -channel blue -separate +channel -compose multiply -composite img_aliasing_area.png
convert img_difference.png -channel red -separate +channel img_aliasing_area.png +matte -compose CopyOpacity -composite -background 

white -compose Over -flatten -normalize  img_maap.png
convert img_maap.png img_aliasing_area.png -compose multiply -composite -negate -background black -channel A  -combine 

img_edging.png
convert img_inside.png img_edging.png -background none -flatten "%file%.png"

echo "%file%" converted.
::Del
   echo Deleting Temporary Files
   del img_difference.png
   del img_outm.png
   del img_inm.png
   del img_inside.png
   del img_aliasing_area.png
   del img_maap.png
   del img_edging.png
   echo Temporary Files Deleted.
:EndDel

But, the program I have is working just fine now--and thanks for all the help!

[anthony]

Okay, I've gotten my question on the subdirectories thing answered elsewhere

Where?

[anthony]

so if you feel like being extremely generous, how would I compact

I'll try... But I may get window syntax wrong, as a am a UNIX programmer.
I do now have a draft UNIX shell version of this, due to this development.

NOTES: the gray(...) should be gray(fuzz_in%%)
or better still a value between in and out fuzz factors.
Basically something between the differences found, so normalize is not
going to interfere with things.

On the line with just a '^'
add ( -clone 3,2,4 -combine -clone 0 +swap +append ^
-write show: +delete ) ^

Code: Select all

rem generate or input a specific background to remove from image
convert "%file%.tif" -fx p{0,0} bg_img.png

pause

rem  image generation order:  (number = parenthesis that generates it)
rem 0 = original image
rem 1 = difference image (blue channel cleared)
rem 2 = outside mask  - definatally transparent
rem 3 = inside mask   - definatally opaque
rem 4 = edge mask     - parts between inside and outside
rem 5 = fully-opaque pixels from original
rem 6 = final image combining all the above
convert "%file%.tif" ^
      ( -clone 0 bg_img.png  -compose Difference -composite ^
         -modulate 100,0  +matte -channel B -evaluate set 0 +channel ) ^
      ( -clone 1 -fill blue -fuzz 2%%  -opaque black ^
         -channel B -separate +channel ) ^
      ( -clone 1 -fill blue -fuzz 4%%  -opaque black ^
         -channel B -separate +channel -negate ) ^
       ( -clone 2,3 -negate -compose multiply -composite ) ^
       ^
       ( -clone 0,3 +matte -compose CopyOpacity -composite ) ^
       ( -clone 1 -channel R -separate +channel ^
         -clone 4 +matte -compose CopyOpacity -composite ^
         -background gray(3%%) -compose Over -flatten -normalize ^
         -clone 4 -compose multiply -composite -negate ^
         -background black -alpha shape ^
         -clone 5 -compose over -composite ) ^
       -delete 0--2  "%file%.png"

echo "%file%" converted.
::Del
   echo Deleting Temporary Files
   del bg_img.png
   echo Temporary Files Deleted.
:EndDel

I have a UNIX shell script for the above, with other options...
http://www.imagemagick.org/Usage/scripts/bg_remove

[illyume]

Okay, I've gotten my question on the subdirectories thing answered elsewhere

Where?

At techsupportforum.com

---

I'll take a look some time at that code you've put up, but for now, the stuff I wrote is running, and working fine.

And, if anyone's interested in knowing, this is what I've got running now:



BatchConvert.bat

Code: Select all

@ECHO OFF
ECHO BatchConvert Program
set /a varfilecount=0
set /a varcount=0
set /p FILEDEL=Delete Original .tif files? (y/n): 
for /F "usebackq delims=" %%k IN (`dir *.tif /s /b`) DO  set /a varfilecount+=1
echo %varfilecount% files found to convert.
pause
for /F "usebackq delims=" %%k IN (`dir *.tif /s /b`) DO  call convertScript.bat "%%~dpnk" %FILEDEL%
Echo %varcount% files have been converted.
PAUSE

along with

convertScript.bat

Code: Select all

::@echo off
::This .bat program requires ImageMagick to be installed on the local computer. www.imagemagick.org
set FILEDEL=%2
set file=%~dpn1


set checkBP="%~dp1"
if %checkBP:~-21,-1%==\Backgrounds-Papers\ goto :convertBGD



set checkBGD="%~n1"
IF %checkBGD:~1,3%==BGD goto :convertBGD 
IF %checkBGD:~1,3%==BGD goto :convertBGD 

echo Converting "%file%.tif"
:Convert Function
   convert "%file%.tif" ( +clone -fx white ) -compose Difference -composite -modulate 100,0  +matte -channel B -evaluate set 0    img_difference.png

   convert img_difference.png -fill blue -fuzz 2%%  -opaque black img_outm.png
   convert img_difference.png -fill blue -fuzz 4%%  -opaque black img_inm.png

   convert img_inm.png -channel blue -separate +channel -negate "%file%.tif" +swap +matte -compose CopyOpacity -composite img_inside.png
   convert img_outm.png -negate img_inm.png -channel blue -separate +channel -compose multiply -composite img_aliasing_area.png
   convert img_difference.png -channel red -separate +channel img_aliasing_area.png +matte -compose CopyOpacity -composite -background white -compose Over -flatten -normalize  img_maap.png
   convert img_maap.png img_aliasing_area.png -compose multiply -composite -negate -background black -channel A  -combine img_edging.png
   convert img_inside.png img_edging.png -background none -flatten "%file%.png"

   echo "%file%" converted.
   :DelTemp
      echo Deleting Temporary Files
      del img_difference.png
      del img_outm.png
      del img_inm.png
      del img_inside.png
      del img_aliasing_area.png
      del img_maap.png
      del img_edging.png
      echo Temporary Files Deleted.
   :EndDelTemp

   IF %FILEDEL%==y (DEL "%file%.tif")
   goto ENDCALL
:EndConvertFunction

:convertBGD
   echo Converting background image "%file%.tif"   
   convert "%file%.tif" "%file%.png"
   IF %FILEDEL%==y (DEL "%file%.tif")
:EndconvertBGD

set /a varcount+=1
:ENDCALL

[illyume]

One more question. It appears that a small number of my images have returned a rather odd error, where somehow the convert process seems to have decided that black should be the transparent color, and not white. The error, I'm pretty sure from everything I'm looking at, seems to be in the function that composes the difference image:

Code: Select all

convert "%file%.tif" ( +clone -fx white ) -compose Difference -composite -modulate 100,0  +matte -channel B -evaluate set 0 img_difference.png

The command prompt window spits out this error when that line is called on these images that aren't working:

Code: Select all

convert: F:\Club_Creatables\Arranged\2004\04-09 Sep\Sunflowers n Scarecrows\Accents\Girl on Sunflower01.tif: unknown field with tag 34016 (0x84e0) encountered. 
`TIFFReadDirectory' @ tiff.c/TIFFWarnings/528.
convert: F:\Club_Creatables\Arranged\2004\04-09 Sep\Sunflowers n Scarecrows\Accents\Girl on Sunflower01.tif: unknown field with tag 34017 (0x84e1) encountered.
`TIFFReadDirectory' @ tiff.c/TIFFWarnings/528.

It spits a similar message to this out about thirty times, and then continues with the rest of the functions, which all work fine, except that the original difference_image ended up with inverted colors from what it should have been. I.E.:


I'm baffled by this. I can't figure out what's going wrong, or how to fix it, at all. Everything else has worked well enough, and the files that aren't working are few enough, that I don't mind too much if I have to just go in and edit these ones by hand later, as I come across them, but it would be nice to have this error fixed, if possible.

Again, though, I am so happily amazed by how nice ImageMagick has made this whole converting images process. Heh, it's saved a -ton- of time, even with the funny hangups here and there.

[fmw42]

perhaps see http://www.imagemagick.org/Usage/formats/#tiff regarding quantum:polarity and your warning about an unknown field. perhaps the tif file is malformed and does not properly specify the quantum:polarity.

I am not an expert on tif, so the IM folks can probably help further. This was just a thought.

[illyume]

Oh, yep. I'm really not sure what the problem is exactly, but I've found it's something wrong with the .tif files--and it looks like I'm not the only one who's had a problem with this error.

So, rather than try to fix it, I've just worked around it. I've adjusted the program so that it first converts everything to .png , and then it runs the transparency functions. Since it ruined a few of my images the first time I ran through everything, I'm going to just start over with my backup copy of all the images, and do all them in this new method.

Heh, this is getting to be a rather long explanation of everything, but I've encountered SO many odd errors and hangups and glitches and stuff! Almost would be a headache, except it's been fun figuring all this out.

[bbstrikesagain]

Hi,

I don't know f this is useful, but I just wrote a .cmd batch file for running convert on dragged files and/or folders. The guide at http://www.imagemagick.org/Usage/windows/ suggests you can't have multiple commands in a For /F iteration, so I see plenty of examples calling further files, but in fact you can group commands in (), as here:

Code: Select all

echo %0
echo Converts files and/or folders of files using ImageMagick...

setlocal
set srcExt=bmp
set tgtExt=png
set tmpDir=tmpdir.tmp
set stdErr=stderr.tmp

echo Source extension: .%srcExt%
echo Target extension: .%tgtExt%
echo.

:loop
if .%1==. goto done
if not exist %1 goto done
mkdir %1\%tmpDir% 2>%stdErr% && goto folder

:file
del %stdErr%
echo Processing file: %1
if "%~x1"==".%srcExt%" convert "%~f1" "%~dpn1.%tgtExt%"
shift
goto loop

:folder
del %stdErr%
rmdir %1\%tmpDir%
echo Parsing folder: %1
%~d1
cd "%~pnx1"
for /F "usebackq delims=" %%i in (`dir /b %1\*.%srcExt%`) do (
  echo Processing file: "%%~nxi"
  convert "%%~nxi" "%%~ni.%tgtExt%" )
shift
goto loop

:done
echo.
echo Done.
echo 
echo.
pause

Sorry if it's slightly off topic, I hit your thread while searching for a problem I'm having with convert.

Okay, I've gotten my question on the subdirectories thing answered elsewhere

Where?

At techsupportforum.com

---

I'll take a look some time at that code you've put up, but for now, the stuff I wrote is running, and working fine.

And, if anyone's interested in knowing, this is what I've got running now:



BatchConvert.bat

Code: Select all

@ECHO OFF
ECHO BatchConvert Program
set /a varfilecount=0
set /a varcount=0
set /p FILEDEL=Delete Original .tif files? (y/n): 
for /F "usebackq delims=" %%k IN (`dir *.tif /s /b`) DO  set /a varfilecount+=1
echo %varfilecount% files found to convert.
pause
for /F "usebackq delims=" %%k IN (`dir *.tif /s /b`) DO  call convertScript.bat "%%~dpnk" %FILEDEL%
Echo %varcount% files have been converted.
PAUSE

along with

convertScript.bat

Code: Select all

::@echo off
::This .bat program requires ImageMagick to be installed on the local computer. www.imagemagick.org
set FILEDEL=%2
set file=%~dpn1


set checkBP="%~dp1"
if %checkBP:~-21,-1%==\Backgrounds-Papers\ goto :convertBGD



set checkBGD="%~n1"
IF %checkBGD:~1,3%==BGD goto :convertBGD 
IF %checkBGD:~1,3%==BGD goto :convertBGD 

echo Converting "%file%.tif"
:Convert Function
   convert "%file%.tif" ( +clone -fx white ) -compose Difference -composite -modulate 100,0  +matte -channel B -evaluate set 0    img_difference.png

   convert img_difference.png -fill blue -fuzz 2%%  -opaque black img_outm.png
   convert img_difference.png -fill blue -fuzz 4%%  -opaque black img_inm.png

   convert img_inm.png -channel blue -separate +channel -negate "%file%.tif" +swap +matte -compose CopyOpacity -composite img_inside.png
   convert img_outm.png -negate img_inm.png -channel blue -separate +channel -compose multiply -composite img_aliasing_area.png
   convert img_difference.png -channel red -separate +channel img_aliasing_area.png +matte -compose CopyOpacity -composite -background white -compose Over -flatten -normalize  img_maap.png
   convert img_maap.png img_aliasing_area.png -compose multiply -composite -negate -background black -channel A  -combine img_edging.png
   convert img_inside.png img_edging.png -background none -flatten "%file%.png"

   echo "%file%" converted.
   :DelTemp
      echo Deleting Temporary Files
      del img_difference.png
      del img_outm.png
      del img_inm.png
      del img_inside.png
      del img_aliasing_area.png
      del img_maap.png
      del img_edging.png
      echo Temporary Files Deleted.
   :EndDelTemp

   IF %FILEDEL%==y (DEL "%file%.tif")
   goto ENDCALL
:EndConvertFunction

:convertBGD
   echo Converting background image "%file%.tif"   
   convert "%file%.tif" "%file%.png"
   IF %FILEDEL%==y (DEL "%file%.tif")
:EndconvertBGD

set /a varcount+=1
:ENDCALL