Foreach command not working with zsh shell on macOS Catalina

1

Hi MRtrix experts,

I’m currently having difficulty executing scripts using the ‘foreach’ batch command on my zsh shell for macOS Catalina (Version 10.15.2)

For example, when importing my diffusion dicom files into a single ‘dwi.mif’ file:

foreach subjects * : mrconvert IN/MY_DICOM_FOLDER IN/dwi.mif

I get the following output: zsh: parse error near `*’

I’m using the latest version of mrtrix3 as per the documentation.

Many Thanks,
Mervyn Singh

2

I don’t have a Mac, so I’m not sure I can help much here. But just a couple of suggestions:

  • should there be a space between subjects and *? Maybe there simply isn’t a single file called subjects, so there’s nothing to match?

  • does this command work if you run it on bash (assuming it’s available…)? I.e. just type bash, then try the exact same command again. If that also fails, it’s not because of zsh.

  • list the contents of your current folder with ls, and verify that there are files in there that should match both subjects and a * wildcard.

  • see what the shell actually expands your full command to: add echo at the start of your command, it’ll show you how the shell has actually interpreted your command before running foreach itself.

3

The issue is that foreach is a shell reserved word in zsh (https://github.com/MRtrix3/mrtrix3/issues/1708). Until we rename the script, you’ll have to use the full path to the script or create an alias for it (alias foreach=$(dirname $(which mrinfo))/foreach). And as Donald mentions, there is a space too many between subjects and *

4

Hi @jdtournier and @maxpietsch,

Thanks so much for your replies! Apologies for the delayed response, I sent this at the awkward period before the long weekend break here in Aus and did not have the chance to reply until I got back to work :slight_smile:

I did a quick google search and found that there is a method of quickly switching the scripting environment from zsh back to bash:

chsh -s /bin/bash

The original website can be viewed here: How to Change the Default Shell to Bash on macOS Catalina

Once the environment has been switched back to bash, the ‘foreach’ command works as per normal. I have previously been successful with using the foreach command with a space between subjects and *

for example:

foreach subjects * mrdegibbs IN/dwi_denoised.mif IN.dwi_denoised_unringed.mif -axes 0,1

I have been running the 'foreach' command previously on bash with a space in between subjects and * and it seems to be executing fine.

While switching the scripting environment works in the interim, I will also create an alias as per @maxpietsch’s advice ( alias foreach=$(dirname $(which mrinfo))/foreach ) so that I can access the command natively in zsh as well.

Many thanks for your help here!

Best,
Mervyn

5

I have previously been successful with using the foreach command with a space between subjects and *
for example:

foreach subjects * : mrdegibbs IN/dwi_denoised.mif IN/dwi_denoised_unringed.mif -axes 0,1

I have been running the 'foreach' command previously on bash with a space in between subjects and * and it seems to be executing fine.

For clarity, while this may work in your specific case, it is still nevertheless indicative of incorrect usage: @jdtournier hinted at this, but with a more technical description.

Let’s say, for instance, you have the following working directory contents:

$ ls
subjects_001/
subjects_002/
$

The purpose of what you type between “foreach” and “:” is to construct the list of elements for which the command after the “:” will be executed. The “*” wildcard is interpreted by the shell and replaced with a list of elements satisfied by that wildcard before foreach is even invoked. In this case, both “subjects_001/” and “subjects_002/” satisfy the wildcard. But the string “subjects” was provided separately to the wildcard, and is therefore its own entry in the list.

So what foreach will actually “see” is:

foreach subjects subjects_001 subjects_002 : ...

It will then perform the appropriate substitutions and execute the following functions:

mrdegibbs subjects/dwi_denoised.mif subjects/dwi_denoised_unringed.mif -axes 0,1
mrdegibbs subjects_001/dwi_denoised.mif subjects_001/dwi_denoised_unringed.mif -axes 0,1
mrdegibbs subjects_002/dwi_denoised.mif subjects_002/dwi_denoised_unringed.mif -axes 0,1

The first of these commands will have failed, since almost certainly no such path exists. However if your list of subject directories is long, you may not have seen the corresponding error message.

The reason the advice of removing that space was given is because if you have more complicated working directory contents, e.g.:

$ ls
subjects_001/
subjects_002/
other.txt
$

, then:

foreach subjects * : ...

becomes:

foreach subjects subjects_001/ subjects_002/ other.txt : ...

; whereas:

foreach subjects/* : ...

becomes:

foreach subjects_001/ subjects_002/ : ...

, which is more likely what you intended.

With the coming update this will be improved: foreach will report the number of inputs for which the corresponding command failed, and print only the terminal outputs of those failed invocations. It also provides a -test option to check your usage and substitutions before running. :+1:

Rob

1 Like
6

Thanks @rsmith for the clarification here! This certainly gives me a better understanding of the foreach command. I didn’t realise I was engaging in incorrect usage and adding redundancies into my workflow :pensive:

I have definitely noticed an error message when the script runs for the first time. I naively did not think much of it since the command would always run successfully after the initial error log. Thanks for pointing me in the right direction :slight_smile:

This is a good point. At present my working directory is not complicated, but it will inevitably be so as I work through my FBA pipeline - so this may be an option I will rely on in the future.

This will be a fantastic addition! I look forward to trying it out when the new update is released :+1:

Once again, thanks for the detailed reply :slight_smile:

Mervyn