There's an algorithm for removing binary noise from an image. I read about it in Practical Algorithms for Image Analysis (a very good book by the way).
It's called KFill
Is there such a beast in ImageMagick?
It looks like just what I need, which is a way to remove smudge like shadows that may appear near edges of scanned documents like ballots etc.