Dwi2response error

1

Hi,

its me again.
I did:

dwi2response tournier data_reg_ecc.mif response.txt

And the error was:

dwi2response: 
dwi2response: Note that this script makes use of commands / algorithms that have relevant articles for citation. Please consult the help page (-help option) for more information.
dwi2response: 
dwi2response: Generated temporary directory: /tmp/dwi2response-tmp-S3EB61/
Command: mrconvert /media/MRTrix_final/data_reg_ecc.mif - -stride 0,0,0,1 | dwiextract - /tmp/dwi2response-tmp-S3EB61/dwi.mif
dwi2response: 
dwi2response: [ERROR] Command failed: mrconvert /media/MRTrix_final/data_reg_ecc.mif - -stride 0,0,0,1 | dwiextract - /tmp/dwi2response-tmp-S3EB61/dwi.mif
dwi2response: Output of failed command:
mrconvert: [ERROR] expected exactly 2 arguments (5 supplied)
dwiextract: [ERROR] no filename supplied to standard input (broken pipe?)
dwiextract: [ERROR] error opening image "-"
dwi2response: Changing back to original directory (/media/MRTrix_final)
dwi2response: Deleting temporary directory /tmp/dwi2response-tmp-S3EB61/`

To be fair…
There is a possibility, that the mif header isnt matching the data in the main part of the file (the number of directions and b-values can be different). But I tried:

dwi2response tournier -grad encoding.b data_reg_ecc.mif response.txt

Doesnt work either.

Thanks,

Ralf

2

Weird. So this is essentially what the problem is reported to be:

mrconvert /media/MRTrix_final/data_reg_ecc.mif - -stride 0,0,0,1 | dwiextract - /tmp/dwi2response-tmp-S3EB61/dwi.mif
mrconvert: [ERROR] expected exactly 2 arguments (5 supplied)

which makes no sense. Can you try running this as-is, see what happens?

mrconvert data_reg_ecc.mif - -stride 0,0,0,1 | dwiextract - tmp-dwi.mif
3

Its working but gives the encoding thing:

rconvert: [100%] copying from "data_reg_ecc.mif" to "/tmp/mrtrix-tmp-aBCt3T.mif"
dwiextract: [ERROR] invalid diffusion gradient table dimensions

With the Gradient table it works:

MRTrix_final$ mrconvert -grad encoding.b data_reg_ecc.mif - -stride 0,0,0,1 | dwiextract - tmp-dwi.mif
mrconvert: [100%] copying from "data_reg_ecc.mif" to "/tmp/mrtrix-tmp-5Fq1or.mif"
dwiextract: [100%] extracting volumes

Would that be the right command:

dwi2response tournier data_reg_ecc.mif -grad encoding.b response.txt

4

Yes, so what I think is going on, is indeed that your first attempt failed due to the header not being correct.
Your original second attempt might have failed due to the scripts needing the arguments first, and only then the options… I think. @rsmith may be able to confirm this one.

Your final suggestion may still be off for that very reason. Maybe try:

dwi2response tournier data_reg_ecc.mif response.txt -grad encoding.b

…i.e., the arguments tournier, data_reg_ecc.mif and response.txt first, and the option -grad encoding.b after that.

If I’m not mistaken, this requirement is due to the argument parsing library that we use for the scripts not being flexible enough. The MRtrix commands, on the other hand, are able to handle any ordering of arguments and options.

EDIT: ok, re-reading the error, I’m not sure the above will be the explanation for that specific error though… it almost seems that in the line…

mrconvert /media/MRTrix_final/data_reg_ecc.mif - -stride 0,0,0,1 | dwiextract - /tmp/dwi2response-tmp-S3EB61/dwi.mif

…it treated everything that did not start with a dash, as an argument (i.e., /media/MRTrix_final/data_reg_ecc.mif, 0,0,0,1, |, dwiextract and /tmp/dwi2response-tmp-S3EB61/dwi.mif) to mrconvert. That’s the only way I can puzzle together “5 supplied” arguments…

5

Your original second attempt might have failed due to the scripts needing the arguments first, and only then the options…

Maybe I’m bashing on argparse too much, it’s not quite that bad… It will accept arguments and options in any order; what it won’t accept is options being provided before the algorithm selector (only applies to 5ttgen and dwi2response currently). This is because the algorithm selector isn’t just ‘the first argument’, but a flag that invokes its own command-line parser. This allows command-line options that are specific to one particular algorithm to only appear in the help page for that particular algorithm, and to be automatically rejected at the command-line if provided for any other algorithm that doesn’t support that option. But anything specified before the argument selector doesn’t get passed along to the relevant sub-parser.

mrconvert: [ERROR] expected exactly 2 arguments (5 supplied)

This may have something to do with the internals of the Python libraries, specifically runCommand(). When a script invokes a command that itself contains a pipe, runCommand() must manually split the commands and run them separately using subprocess. Something might be going wrong in there.

Firstly, can you make sure that MRtrix3 is fully up-to-date (since this has changed a bit in the recent past). Then, change this empty line to:
print (cmdstack)
That will tell me exactly what the system is trying to execute, and therefore whether or not something is going wrong during this process.

6

Hi and thanks for your help.

  1. There were no updates to make. Nice. :slight_smile:
  2. I filled in the line at the same column like the if-condition. (I think in python its column sensitiv, but I am not sure.)
  3. I didnt compile. Right? Python hasnt to?!

Codeline to start:

ibmi@ibmi:/MRTrix_final$ dwi2response tournier data_reg_ecc.mif response.txt -grad encoding.b

Error MEssage:

dwi2response: 
    dwi2response: Note that this script makes use of commands / algorithms that have relevant articles for citation. Please consult the help page (-help option) for more information.
    dwi2response: 
    dwi2response: Generated temporary directory: /tmp/dwi2response-tmp-XHHZJ9/
    Command: mrconvert /MRTrix_final/data_reg_ecc.mif - -stride 0,0,0,1 -grad /MRTrix_final/encoding.b | dwiextract - /tmp/dwi2response-tmp-XHHZJ9/dwi.mif
    [['mrconvert', '/MRTrix_final/data_reg_ecc.mif', '-', '-stride', '0,0,0,1', '-grad', '/MRTrix_final/encoding.b', '-quiet'], ['dwiextract', '-', '/tmp/dwi2response-tmp-XHHZJ9/dwi.mif', '-quiet']]
    dwi2response: 
    dwi2response: [ERROR] Command failed: mrconvert /MRTrix_final/data_reg_ecc.mif - -stride 0,0,0,1 -grad /MRTrix_final/encoding.b | dwiextract - /tmp/dwi2response-tmp-XHHZJ9/dwi.mif
    dwi2response: Output of failed command:
    mrconvert: [ERROR] expected exactly 2 arguments (8 supplied)
    dwiextract: [ERROR] no filename supplied to standard input (broken pipe?)
    dwiextract: [ERROR] error opening image "-"
    dwi2response: Changing back to original directory (/MRTrix_final)
    dwi2response: Deleting temporary directory /tmp/dwi2response-tmp-XHHZJ9/
    ibmi@ibmi:/MRTrix_final$

I hope you find a solution. If not I have to rewrite my code and have to find a way to change the header. (Is there a way in MRTrix to reorient (rotate with a transfomation matrix the Diffusion Vectors after flirt or other registrations?)

Ralf

PS: If it helps. I can shorten the message and del the Folder lines.

7

Yep, that gave me the information I was looking for. And correct: no need to recompile for changes to Python scripts.

So we’ve got these two errors:

Command: mrconvert /media/MRTrix_final/data_reg_ecc.mif - -stride 0,0,0,1 -quiet
mrconvert: [ERROR] expected exactly 2 arguments (5 supplied)

mrconvert /MRTrix_final/data_reg_ecc.mif - -stride 0,0,0,1 -grad /MRTrix_final/encoding.b -quiet
mrconvert: [ERROR] expected exactly 2 arguments (8 supplied)

, and runCommand() seems to be behaving as it should. It’s almost as though opening dashes from options are being ‘split’ from the option strings… but even then the numbers don’t come out right. I think we’re going to have to do GitHub issue 536, so that we can see what’s actually being provided to mrconvert. I’ll let you know when there’s an update for you to install and test; might not be until tomorrow though.

P.S. Don’t suppose you’ve had a change of Python version recently? Just on the off chance that they’ve introduced a bug in subprocess? :confused:

8

Hi,

which Python version should I install? Can I test something or update python?

Ralf

9

Really confusing… I can’t see the image headers having anything to do with this, the error occurs at the argument parsing stage, before mrconvert would even attempt to access the images.

One thing to think about is the potential impact of locale settings - they have caused funny issues in the past. I have to say, I can’t see how the script would change the ASCII dash to something else, but if it did, that could explain why it’s not being recognised in the binaries…? Although even in this case, the numbers don’t seem to add up…

@Ralf, any chance you could post the output of the locale command on your system…?

You could also try to reset your locale settings like this:

unset LANG LC_CTYPE LC_NUMERIC LC_TIME LC_COLLATE LC_MONETARY LC_MESSAGES LC_PAPER LC_NAME LC_ADDRESS LC_TELEPHONE LC_MEASUREMENT LC_IDENTIFICATION LC_ALL

before trying your dwi2response command again. If that still doesn’t work, maybe try using the standard ‘C’ locale:

export LANG=C LC_ALL=C

and try dwi2response again…

10

Hier are the Locales:

ibmi@ibmi-81645:~$ locale
LANG=de_DE.UTF-8
LANGUAGE=de_DE
LC_CTYPE="de_DE.UTF-8"
LC_NUMERIC="de_DE.UTF-8"
LC_TIME="de_DE.UTF-8"
LC_COLLATE="de_DE.UTF-8"
LC_MONETARY="de_DE.UTF-8"
LC_MESSAGES="de_DE.UTF-8"
LC_PAPER="de_DE.UTF-8"
LC_NAME="de_DE.UTF-8"
LC_ADDRESS="de_DE.UTF-8"
LC_TELEPHONE="de_DE.UTF-8"
LC_MEASUREMENT="de_DE.UTF-8"
LC_IDENTIFICATION="de_DE.UTF-8"
LC_ALL=
ibmi@ibmi-81645:~$
11

I use this command:

dwi2response tournier data_reg_ecc.mif response.txt -grad encoding.b

for all the tests.

Resetting the locals didnt help and export didnt help too.

Sorry.

12

I also took the DWI data right after the mrconvert from Dicom. Without the changed data and its the same.

dwi2response tournier DWI_1.4mmiso_3000bh27_3631_B3000_1.4iso.mif response.txt

Error:

dwi2response: 
dwi2response: Note that this script makes use of commands / algorithms that have relevant articles for citation. Please consult the help page (-help option) for more information.
dwi2response: 
dwi2response: Generated temporary directory: /tmp/dwi2response-tmp-JHDCZV/
Command: mrconvert /media/ibmi/Data/Paper_DWI_Treffen_2016.07.20_Miitwoch/Paper_DWI_Vergleich_B1000-3000 bei 1.4mm iso/bh27_B1000_neu/MRTrix_final/test/DWI_1.4mmiso_3000bh27_3631_B3000_1.4iso.mif - -stride 0,0,0,1 | dwiextract - /tmp/dwi2response-tmp-JHDCZV/dwi.mif
[['mrconvert', '/media/ibmi/Data/Paper_DWI_Treffen_2016.07.20_Miitwoch/Paper_DWI_Vergleich_B1000-3000', 'bei', '1.4mm', 'iso/bh27_B1000_neu/MRTrix_final/test/DWI_1.4mmiso_3000bh27_3631_B3000_1.4iso.mif', '-', '-stride', '0,0,0,1', '-quiet'], ['dwiextract', '-', '/tmp/dwi2response-tmp-JHDCZV/dwi.mif', '-quiet']]
dwi2response: 
dwi2response: [ERROR] Command failed: mrconvert /media/ibmi/Data/Paper_DWI_Treffen_2016.07.20_Miitwoch/Paper_DWI_Vergleich_B1000-3000 bei 1.4mm iso/bh27_B1000_neu/MRTrix_final/test/DWI_1.4mmiso_3000bh27_3631_B3000_1.4iso.mif - -stride 0,0,0,1 | dwiextract - /tmp/dwi2response-tmp-JHDCZV/dwi.mif
dwi2response: Output of failed command:
mrconvert: [ERROR] expected exactly 2 arguments (5 supplied)
dwiextract: [ERROR] no filename supplied to standard input (broken pipe?)
dwiextract: [ERROR] error opening image "-"
dwi2response: Changing back to original directory (/media/ibmi/Data/Paper_DWI_Treffen_2016.07.20_Miitwoch/Paper_DWI_Vergleich_B1000-3000 bei 1.4mm iso/bh27_B1000_neu/MRTrix_final/test)
dwi2response: Deleting temporary directory /tmp/dwi2response-tmp-JHDCZV/`
13

Yes, the issue is completely unrelated to the data you’re trying to process, it seems that the script is issuing malformed commands for some reason, which the actual executables don’t understand. We just can’t figure out why…

14

Okay. I´ll wait. :slight_smile:

15

OK @Ralf, I’ve just merged some changes that should give us some more information as to what on earth is going on here. Update, recompile and have another go?

16

Good Morning,

okay…I will try.

17

Without -grad:

ibmi@ibmi-81645:/media/ibmi/Data/Paper_DWI_Treffen_2016.07.20_Miitwoch/Paper_DWI_Vergleich_B1000-3000 bei 1.4mm iso/bh27_B1000_neu/MRTrix_final/test$ dwi2response tournier DWI_1.4mmiso_3000bh27_3631_B3000_1.4iso.mif response.txt
dwi2response: 
dwi2response: Note that this script makes use of commands / algorithms that have relevant articles for citation. Please consult the help page (-help option) for more information.
dwi2response: 
dwi2response: Generated temporary directory: /tmp/dwi2response-tmp-J3BN6J/
Command: mrconvert /media/ibmi/Data/Paper_DWI_Treffen_2016.07.20_Miitwoch/Paper_DWI_Vergleich_B1000-3000 bei 1.4mm iso/bh27_B1000_neu/MRTrix_final/test/DWI_1.4mmiso_3000bh27_3631_B3000_1.4iso.mif - -stride 0,0,0,1 | dwiextract - /tmp/dwi2response-tmp-J3BN6J/dwi.mif
[['mrconvert', '/media/ibmi/Data/Paper_DWI_Treffen_2016.07.20_Miitwoch/Paper_DWI_Vergleich_B1000-3000', 'bei', '1.4mm', 'iso/bh27_B1000_neu/MRTrix_final/test/DWI_1.4mmiso_3000bh27_3631_B3000_1.4iso.mif', '-', '-stride', '0,0,0,1', '-quiet'], ['dwiextract', '-', '/tmp/dwi2response-tmp-J3BN6J/dwi.mif', '-quiet']]
dwi2response: 
dwi2response: [ERROR] Command failed: mrconvert /media/ibmi/Data/Paper_DWI_Treffen_2016.07.20_Miitwoch/Paper_DWI_Vergleich_B1000-3000 bei 1.4mm iso/bh27_B1000_neu/MRTrix_final/test/DWI_1.4mmiso_3000bh27_3631_B3000_1.4iso.mif - -stride 0,0,0,1 | dwiextract - /tmp/dwi2response-tmp-J3BN6J/dwi.mif
dwi2response: Output of failed command:
mrconvert: [ERROR] Expected exactly 2 arguments (5 supplied)
mrconvert: [ERROR] Usage: mrconvert input output
mrconvert: [ERROR] Yours: mrconvert /media/ibmi/Data/Paper_DWI_Treffen_2016.07.20_Miitwoch/Paper_DWI_Vergleich_B1000-3000 bei 1.4mm iso/bh27_B1000_neu/MRTrix_final/test/DWI_1.4mmiso_3000bh27_3631_B3000_1.4iso.mif -
dwiextract: [ERROR] no filename supplied to standard input (broken pipe?)
dwiextract: [ERROR] error opening image "-"
dwi2response: Changing back to original directory (/media/ibmi/Data/Paper_DWI_Treffen_2016.07.20_Miitwoch/Paper_DWI_Vergleich_B1000-3000 bei 1.4mm iso/bh27_B1000_neu/MRTrix_final/test)
dwi2response: Deleting temporary directory /tmp/dwi2response-tmp-J3BN6J/`

With -grad:

ibmi@ibmi-81645:/media/ibmi/Data/Paper_DWI_Treffen_2016.07.20_Miitwoch/Paper_DWI_Vergleich_B1000-3000 bei 1.4mm iso/bh27_B1000_neu/MRTrix_final$ dwi2response tournier data_reg_ecc.mif response.txt -grad encoding.b
dwi2response: 
dwi2response: Note that this script makes use of commands / algorithms that have relevant articles for citation. Please consult the help page (-help option) for more information.
dwi2response: 
dwi2response: Generated temporary directory: /tmp/dwi2response-tmp-PZ57QN/
Command: mrconvert /media/ibmi/Data/Paper_DWI_Treffen_2016.07.20_Miitwoch/Paper_DWI_Vergleich_B1000-3000 bei 1.4mm iso/bh27_B1000_neu/MRTrix_final/data_reg_ecc.mif - -stride 0,0,0,1 -grad /media/ibmi/Data/Paper_DWI_Treffen_2016.07.20_Miitwoch/Paper_DWI_Vergleich_B1000-3000 bei 1.4mm iso/bh27_B1000_neu/MRTrix_final/encoding.b | dwiextract - /tmp/dwi2response-tmp-PZ57QN/dwi.mif
[['mrconvert', '/media/ibmi/Data/Paper_DWI_Treffen_2016.07.20_Miitwoch/Paper_DWI_Vergleich_B1000-3000', 'bei', '1.4mm', 'iso/bh27_B1000_neu/MRTrix_final/data_reg_ecc.mif', '-', '-stride', '0,0,0,1', '-grad', '/media/ibmi/Data/Paper_DWI_Treffen_2016.07.20_Miitwoch/Paper_DWI_Vergleich_B1000-3000', 'bei', '1.4mm', 'iso/bh27_B1000_neu/MRTrix_final/encoding.b', '-quiet'], ['dwiextract', '-', '/tmp/dwi2response-tmp-PZ57QN/dwi.mif', '-quiet']]
dwi2response: 
dwi2response: [ERROR] Command failed: mrconvert /media/ibmi/Data/Paper_DWI_Treffen_2016.07.20_Miitwoch/Paper_DWI_Vergleich_B1000-3000 bei 1.4mm iso/bh27_B1000_neu/MRTrix_final/data_reg_ecc.mif - -stride 0,0,0,1 -grad /media/ibmi/Data/Paper_DWI_Treffen_2016.07.20_Miitwoch/Paper_DWI_Vergleich_B1000-3000 bei 1.4mm iso/bh27_B1000_neu/MRTrix_final/encoding.b | dwiextract - /tmp/dwi2response-tmp-PZ57QN/dwi.mif
dwi2response: Output of failed command:
mrconvert: [ERROR] Expected exactly 2 arguments (8 supplied)
mrconvert: [ERROR] Usage: mrconvert input output
mrconvert: [ERROR] Yours: mrconvert /media/ibmi/Data/Paper_DWI_Treffen_2016.07.20_Miitwoch/Paper_DWI_Vergleich_B1000-3000 bei 1.4mm iso/bh27_B1000_neu/MRTrix_final/data_reg_ecc.mif - bei 1.4mm iso/bh27_B1000_neu/MRTrix_final/encoding.b
dwiextract: [ERROR] no filename supplied to standard input (broken pipe?)
dwiextract: [ERROR] error opening image "-"
dwi2response: Changing back to original directory (/media/ibmi/Data/Paper_DWI_Treffen_2016.07.20_Miitwoch/Paper_DWI_Vergleich_B1000-3000 bei 1.4mm iso/bh27_B1000_neu/MRTrix_final)
dwi2response: Deleting temporary directory /tmp/dwi2response-tmp-PZ57QN/
ibmi@ibmi-81645:/media/ibmi/Data/Paper_DWI_Treffen_2016.07.20_Miitwoch/Paper_DWI_Vergleich_B1000-3000 bei 1.4mm iso/bh27_B1000_neu/MRTrix_final$

I will change the folder more down the tree for future Replies.

Question: Di I have to reboot?

Hope u find something,

Ralf

18

OK, clearly the script is passing the full path to the current folder, but not handling spaces in the folder name appropriately. Your current folder is /media/ibmi/Data/Paper_DWI_Treffen_2016.07.20_Miitwoch/Paper_DWI_Vergleich_B1000-3000 bei 1.4mm iso/bh27_B1000_neu/MRTrix_final/test/ - which has 3 spaces in it. This gets used in the script, and split into 4 arguments rather than one. There’s nothing you did wrong here, we just need to handle that case appropriately in the script.

Thanks for investing so much time debugging this, we’ll get it fixed ASAP.

19

I see. Didnt noticed it. I never use spaces. I use the " _ " instead.
Thats why. :slight_smile:Unbelievable.

Thanks you.

20

@Ralf: you shouldn’t need to worry about using spaces. They need a bit more care in handling, but it should be just fine to use them. We just need to fix the script.

Just a quick follow up on this: this was actually evident from @Ralf’s previous output, can’t believe we didn’t spot it then. But it doesn’t seem to tally with any of the posts before that, which were all run from /media/MRtrix_final: no spaces there, and the additional debug output that @rsmith suggested didn’t show this to be the problem - yet the symptoms were identical (same number of arguments provided in each case). I’m struggling to understand what the issue might have been, assuming it must be related. Maybe that folder was actually a symbolic link / shortcut to the longer named folder with spaces in? In which case maybe the script dereferences the link before actually running the command?

@Ralf, what system are you running this on? I could imagine Windows’ handling of symbolic links might cause trouble like this, but I find it harder to rationalise on a true UNIX system…