Unix pipes - using the same temporary file twice?

Quick question - is it possible to use the same temporary file twice within a piped command?

I tried something like this (extract a ROI based on some parcellation, then crop to prevent FPS from tanking in volumetric view), but it results in an error:

mrcalc test.mif 5 -eq - -datatype bit | mrgrid - crop -mask - test.mif                                                             
mrcalc: [100%] computing: (test.mif == 5)
mrgrid: [ERROR] no filename supplied to standard input (broken pipe?)
mrgrid: [ERROR] error opening image "-"

Is the tmp file being deleted right after being used as the first argument to mrgrid? Or is there some error in the command / did I misunderstand piping?

mrcalc test.mif 5 -eq - -datatype bit | mrinfo -  

works without problems.

Thanks!

Hi @PabloPretzel,

OK, this is going to require a fairly deep dive into how we use Unix pipes in MRtrix… It’s all explained in the docs, but the main point is that the first command prints out the filename of a temporary file to standard output, and the next command reads the filename from standard input – the shell is responsible for plumbing the standard output of the first command to the standard input of the second.

In your case, you want to use the same (temporary) file twice, as arguments that would in the general case not be the same (the image being cropped is rarely the same as the mask from which the crop region is computed). What happens is that the temporary filename is printed once by mrcalc, but mrgrid tries to read two filenames from the pipe – one for each argument¹. But there’s only one filename to read, hence the error when trying to open the second image.

Technically, you can do what you want, but it requires some slightly ugly hacks…


Option 1:

Insert an additional stage in the pipeline to duplicate each line in the stream. There’s probably other ways to do this, but this one does the trick:

mrcalc test.mif 5 -eq - -datatype bit | awk '{print $0; print $0;}' | mrgrid - crop -mask - test.mif                                                             

(I said it was ugly…)


Option 2:

Capture the filename of the temporary output as a variable, then substitute this variable into the next command instead of the - placeholder. This behaves the same as if it had been piped, but uses the filename as supplied on the command-line rather than reading it from standard input:

tmp=$(mrcalc test.mif 5 -eq - -datatype bit) && mrgrid "$tmp" crop -mask "$tmp" test.mif                                                             

I’ve used the && control operator to make the execution of the second command conditional on successful completion of the first.

Personally, I prefer the second option as it gives you more control and visibility into what’s going on, but it’s still not pretty…


Note also that this is likely to work only on Unix-like systems, since mrgrid will open the same file twice, and then delete the file twice. Unix handles this gracefully by unlinking from the filesystem (some context here if you’re interested). In my experience, Windows is much stricter about deleting open files, and I suspect this may be an instance where this approach might fail.

All the best,
Donald.


¹ As as aside, the order in which these are read from the pipe is not guaranteed to match the order of the arguments on the command line – it depends on which file the command is programmed to access first. Not an issue in your case since they’re the same file anyway…

2 Likes

Hi Donald,

wow, thanks for taking your time to answer such a niche question in such depth! Didn’t know you hold a computer science / OS design PhD in addition to all your other credentials :smiley:

All the best,
Pablo

2 Likes