I recently came across two unexpected behaviors of mrTransform. I’m not sure whether they’re bugs of if I’m reading the help page incorrectly:
Forcing rewrite on an output file that’s identical to the the input file results in an empty image. For instance: mrtransform dwi.mif -linear transform.txt dwi.mif -force
The output of this (dwi.mif) has the expected transformation matrix, but the image is all zeros.
Using the -replace option to copy the transformation matrix from one file to another results in an error: mrtransform dwi1.mif -replace dwi2.mif dwi3.mif mrtransform: [ERROR] Expected exactly 2 arguments (3 supplied) mrtransform: [ERROR] Usage: mrtransform input output
The same happens if trying to use the -replace option with an ascii file.
I just tried both of those cases, I can’t reproduce these errors. Everything works as expected on my end. What platform are you running on? This might have a bearing on the first issue: if you’re running on Windows, it might be that having the file open (e.g. in MRView) while running the second (overwrite) command could be problematic, due to Windows’ strict single concurrent access rules. i certainly don’t see this issue here on Linux.
For the second issue, I’ve no idea why this might not be working for you, other than your install potentially being out of date?
The first case we generally recommend should be avoided whenever possible. Even if it can be used in certain scenarios, it’s not a good habit to get into. It could fail depending on e.g. whether files are memory-mapped, whether threading is involved in the copy, or whether it’s images or some other form of data being manipulated (e.g. trying to do this in tckedit would be guaranteed to fail).
Previously, a transformation would be provided using the -linear option, and then the -replace ‘flag’ would indicate that the transform provided using -linear should ‘replace’ the header transform rather than being ‘applied’. I found this clunky, so made modifications:
Now, the -linear and -replace options are separate. Therefore -replace requires that a transformation text file or template image be provided alongside it.
You’re getting that particular error message because the -replace option is not ‘claiming’ the next item on the command-line. Therefore mrtransform sees dwi1.mif, dwi2mif and dwi3.mif as all being arguments, despite your expectation that (-replace dwi2.mif) behave as an option and therefore mrtransform should only see dwi1.mif and dwi3.mif as arguments.
Updating and re-compiling MRtrix3 should get the behaviour back in line with the up-to-date online documentation.
Woops, sorry! I’d failed to spot you were trying to overwrite the input image there. @rsmith is spot on, you should never do this. MRtrix3 is generally designed to avoid loss of data whenever possible, which is why we don’t allow overwrite by default without the -force option. When you do use it though, we skip all these checks, but it doesn’t mean that the operation is well-defined. The main issue here is that when MRtrix3 creates a new image, it will initialise the data with zeros. This is to make sure the output is well-defined even if the operation only applies to selected voxels within a mask, for example. So the very first thing that will happen when the command tries to create the output (overwriting the input), it to fill it with zeros - and since the images are accessed via memory-mapping, in your case that also zeros the input image - before it can even start copying the data over.
So this command can only succeed in very select circumstances where the data is preloaded into RAM before the output image is created. Knowing when this is a safe assumption requires some pretty good knowledge of the image format handling and the particular command’s behaviour. MRtrix3 will try very hard not to preload when memory mapping can be used without loss of performance (it’s generally much more efficient and reduces RAM usage). Some commands may explicitly try to preload to speed up subsequent processing, but again MRtrix3 will only preload when actually necessary (notable the datatype or strides don’t match the command’s expectations). So it’s very difficult to know for certain when this would happen.
You may ask why we don’t therefore check whether the input is the same as the output. That’s a fair question, and the answer is it’s actually not as simple as it sounds. We can check whether the filenames match, but that’s very fragile (won’t catch dwi.mif → ./dwi.mif for example, or when the same image is accessed via symbolic or hard links). So we’d need to rely on the OS’s mechanisms for checking concurrent access to the same actual file, and that’s not as simple as it sounds, and most likely not portable either. Besides, I’m not sure there’s any mechanism in Linux to query whether a particular file is already being accessed in the same process. When I’d looked into this previously, the most likely mechanism was file-locking, which brings in quite a few additional problems and isn’t portable to other OS’s (see here for an overview of the problems). And then it gets more complicated again for multi-file formats… It’s easy to check whether the output files exist, so we can provide non-overwrite guarantees that way. But when you’re using the -force option, checking whether we’re already accessing this image in the same process is unfortunately not easy.
Long story short: don’t do this in any MRtrix3 command, it’s not safe…
Indeed, updating mrTrix fixed the problem with the -reverse option. I guess that should always be the first thing to try…
As for the force rewrite, I didn’t think it would be a problem because I thought I was only changing the header and not that the whole file was being overwritten. It would be nice to see a warning that this particular call is most likely going to erase the data (for people who don’t know the details of the implementation), but I understand the difficulty. I won’t try this again, I promise.
Totally agree. I think I may have a way to detecting such cases, I’ll make a note of it to make sure it doesn’t fall off the radar.
But just to add yet more detail on this issue: you’re right that this is a header-only modification, but unfortunately that doesn’t guarantee that the data won’t be affected in general. It’s fine for NIfTI images since that uses a fixed-size header, but .mif files have a variable size header. Any change to its size might mean that the data portion starts from a different byte offset, so it’s not possible to guarantee that a header-only modification won’t also require shifting the data about to accommodate…