There is a physical basis to wanting to downsize in linear ("physical") light, as opposed to perceptual (sRGB et al) light: You want downsampling to mimick the physical process of "averaging" photons hitting one spot of your retina, and this averaging is almost certainly better modeled by averaging linear light values.
One may be tempted to infer that enlarging (upsampling) should also be done through linear light. I certainly made this inference myself. And it is wrong.
Although I have not checked things with sufficiently many ways of enlarging through linear light (using both different conversion methods and different linear light colourspaces), I feel pretty confident that the following statement will not come back to haunt me:
(The same probably holds with Adobe RGB and some of the other "perceptual" colourspaces.)(Nicolas') Conjecture
If you are enlarging (specifically, if you are not significantly downsampling) an sRGB image with a filter that has significant negative lobes (like Lanczos or Catmull-Rom), do not go through linear light, even if your set up allows you to convert between sRGB and linear light with insignificant loss (with IM, anything but an 8-bit compile if you use linear RGB with sRGB primaries; with XYZ, stick to HDRI). You are better off resampling with "straight" sRGB values.
Short version: With a sharpening filter (Lanczos, Catmull-Rom etc), don't enlarge in linear light.
P.S. There has to be someone who noticed this before. If you have references (web, books or articles), please communicate them to me.
P.S. The above assumes that your image is reasonably close to being a "natural image" in which sharply focused foreground objects are generally darker than their background. If your sRGB image is better described as being "reverse video", you may or may not (I don't really know) be better off enlarging in linear light (or, at least, enlarging with a "positive" version of the image). An attempt at dealing with this caveat is discussed here: viewtopic.php?f=22&t=21415.
P.S. These conclusions were made comparing a "straight" IM enlarge with one that goes through linear RGB with sRGB primaries using the IM7 -colorspace command. At some point, I'll need to make sure that they are not specific to the comparison of these two toolchains.
P.S. I forgot to thank my graduate student Adam Turcotte. The above conjecture was formulated while trying to make sense of the results that are coming out of his Masters thesis work in progress EXQUIRES (https://github.com/aturcotte/exquires).
)
