Dwi2response with msmt_5tt HCP tutorial


#1

Hi all,

this morning I had to re-install mrtrix and now the HCP tutorial, that worked smoothly before, get stuck. The problem is the Diffusion Processing, the dwi2response bit:

this my command
dwi2response msmt_5tt DWI.mif 5TT.mif RF_GM.txt RF_WM.txt RF_CSF.txt -voxels RF_voxels.mif -force

here the errors (everything seems to work fine for the dwi2mask, the mrtrasform and the mrconvert steps):



dwi2response: Calling dwi2response recursively to select WM single-fibre voxels using ‘tournier’ algorithm
Command: dwi2response tournier dwi.mif wm_ss_response.txt -mask wm_mask.mif -voxels wm_sf_mask.mif -quiet -tempdir /tmp/dwi2response-tmp-AZWB5H/
dwi2response:
dwi2response: [ERROR] Command failed: dwi2response tournier dwi.mif wm_ss_response.txt -mask wm_mask.mif -voxels wm_sf_mask.mif -quiet -tempdir /tmp/dwi2response-tmp-AZWB5H/
dwi2response: Output of failed command:
dwi2response: Script failed while executing the command: fixel2voxel iter0_peaks.msf split_value iter0_amps.mif
dwi2response: For debugging, inspect contents of temporary directory: /tmp/dwi2response-tmp-AZWB5H/dwi2response-tmp-1CLADS/
dwi2response: [ERROR] Command failed: fixel2voxel iter0_peaks.msf split_value iter0_amps.mif
dwi2response: Output of failed command:
fixel2voxel: [ERROR] fixel image is empty
dwi2response: Changing back to original directory (~/HCP/100307/T1w)
dwi2response: Script failed while executing the command: dwi2response tournier dwi.mif wm_ss_response.txt -mask wm_mask.mif -voxels wm_sf_mask.mif -quiet -tempdir /tmp/dwi2response-tmp-AZWB5H/
dwi2response: For debugging, inspect contents of temporary directory: /tmp/dwi2response-tmp-AZWB5H/

Thanks!!


Problem with dwi2response script
#2

I’ve tried again, running “dhollander” instead of msmt_5tt and that step worked (dwi2response dhollander DWI.mif RF_WM.txt RF_GM.txt RF_CSF.txt -voxels RF_voxels.mif -force).

Wondering maybe it could be an issue due to the update of the dwi2response itself? Now not working anymore with msmt_5tt?

Sara


#3

Hi Sara,

Could you check whether the prior 5 tissue type segmentation worked as expected? If you inspect 5TT.mif using mrview, do the different tissue types show up as expected?


#4

Hi Ben,

you are right, I had a look at the 5TT and it didn’t work. Now I have to check why but for the dwi2response is solved :slight_smile: Thanks!


#5

Hi @sara ,

Thanks for giving my “dhollander” algorithm a shot in the mean time as well. :smiley:
I’ve tested it on a wide range of data (including HCP data), and it should generalise pretty well; so you can definitely use it too for this application. In fact, we’ve discovered that there’s quite a lot of cases where it ends up even doing a better job than the msmt_5tt algorithm (due to the latter relying on a very, very (very) accurate coregistration across the entire brain between the T1 and dMRI datasets). If you’d like more information on it, here’s the details: https://www.researchgate.net/publication/307863133_Unsupervised_3-tissue_response_function_estimation_from_single-shell_or_multi-shell_diffusion_MR_data_without_a_co-registered_T1_image

It’s still very new though (I presented it yesterday here at the ISMRM diffusion workshop in Lisbon); so I always love hearing about your experiences with it!

Cheers,
Thijs


#6

Hi Thijs,

I’m actually pretty satisfied with the new “dhollander” algorithm (it also quite fast! And that is a great advantage). Till now I’ve been using it just with multi-shell (HCP) data but I’m planning to try using it with single-shell data.
What it feel now as the major difference between “dhollander” and “msmt_5tt” is that either msmt_5tt slightly underestimated the wm fod or dhollander slightly overestimated it. Was wondering if it could be related to the fact that the dhollander is a unsupervised algorithm - I kind of remember you can have overestimation in the unsupervised learning, but It’s not my field at all :slight_smile:
Thanks again!
Sara


#7

Thanks for reporting back @sara! I was just wondering what you specifically meant with this: I’m slightly confused because you say “WM FOD” in this context (and those algorithms just extract responses). Are you referring to the FODs after applying MSMT-CSD, while using the responses from these algorithms? If so, it may be that you’re describing the effect that I saw in the abstract?

In that case, the funny thing is that this is actually (in a certain way) because the “dhollander” algorithm is quite “conservative”; so the responses are (most of the time) a bit more “pure”, and so they sit at wider extremes and allow a slightly wider spectrum of things to be successfully fitted by the multi-tissue CSD thereafter. There’s different ways to reason about this though (as well as what you’d actually want to expect from a multi-tissue CSD outcome in general). It’s not always trivial what is “best” here: it may very well depend on what you’re after in terms of interpretation of the final result. (I’m sorry if this paragraph is slightly cryptic; in that case, feel free to ignore it :smile:)

But in any way: thanks a lot for reporting back. I’m always interested to hear about peoples’ experiences with this algorithm!

Cheers,
Thijs


#8

Just as an extra comment: while you can indeed already use this algorithm for 3-tissue response function estimation in single shell data, note that you can’t do much with these responses yet thereafter: the MSMT-CSD method still requires at least 3 b-values (including the b=0 data) if you want to separate WM, GM and CSF tissue classes. The best you could do on that front at the moment for single-shell data, would be to ask the MSMT-CSD algorithm just for, e.g., the WM and the CSF (and ditch the GM response). That would at least already do a (suboptimal) form of free-water elimination (and a-tiny-bit-of-GM-elimination, due to a slight bias).

That’s all about to change (hopefully not too long from now) though; I’ll provide an implementation of single-shell 3-tissue CSD at some point.


#9

Yes, exactly! Sorry, re-reading my comment, was actually quite lacking of details :slight_smile:
You are right, I was referring to what I can see after applying dhollander or msmt_5tt AND the msmt_csd thereafter. And it seems exactly what you observed in your abstract, cool!

Thanks again,
cheers

Sara


#10

Hooraay for reproducible research!! :sunglasses: :smile:

Thanks for confirming; this is really helpful! :thumbsup:


#11

@Thijs, Unfortunately, I failed using the parameter 'dhollander '. When run the script:

dwi2response dhollander DWI.mif RF_WM.txt RF_GM.txt RF_CSF.txt

It produced more than 10GB temptory files before output errors:

Command: mrcalc _crudegmhighselect.mif 1 _crudegmlowselect.mif -if refined_gm.mif -datatype bit
Command: mrcalc _crudewmoutliers.mif safe_sdm.mif 0 -if 1.93704 -gt 1 crude_csf.mif -if _crudecsfextra.mif -datatype bit
Command: mrcalc _crudecsfextra.mif safe_sdm.mif 1.93704 -subtract 0 -if - | mrthreshold - - -mask _crudecsfextra.mif | mrcalc _crudecsfextra.mif - 0 -if refined_csf.mif -datatype bit
dwi2response: Running 'tournier' algorithm to select 1619 single-fibre WM voxels.
Command: dwi2response tournier dwi.mif _respsfwmss.txt -sf_voxels 1619 -iter_voxels 16190 -mask refined_wm.mif -voxels voxels_sfwm.mif -quiet -tempdir /tmp/dwi2response-tmp-P8M1DI/
dwi2response:
dwi2response: [ERROR] Command failed: dwi2response tournier dwi.mif _respsfwmss.txt -sf_voxels 1619 -iter_voxels 16190 -mask refined_wm.mif -voxels voxels_sfwm.mif -quiet -tempdir /tmp/dwi2response-tmp-P8M1DI/
dwi2response: Output of failed command:

 dwi2response: [ERROR] unknown option "-sf_voxels"**

PS: The multi-shell DWI data is HCP 100048
DWI.mif was produced by
mrconvert …/T1w/Diffusion/data.nii.gz DWI.mif -fslgrad …/T1w/Diffusion/bvecs …/T1w/Diffusion/bvals -datatype float32 -stride 0,0,0,1

Thanks


#12

Hi @Haiyong_Wu,

It looks like the problem you’re running into is due to remnants of an older MRtrix version still being there. I suppose you’ve been an MRtrix user since before the big update we did in March?

The problem is that you still may have a command called dwi2response sitting around, and that is causing conflicts with our new dwi2response script.

If this is the case, try running the following at the command line:

./build clean
git pull
./configure
./build

…and see if that fixes things. This should clean all the old stuff up, get the latest version of everything, and set everything up fresh.

Cheers,
Thijs


#13

Hey @Thijs!!

running “dhollander” algorithm for single shell I found this problem :confused:

dwi2response: [ERROR] Command failed: mrcalc _crudenonwm.mif safe_sdm.mif  -subtract 0 -if - | mrthreshold - - -mask _crudenonwm.mif | mrcalc _crudenonwm.mif - 0 -if crude_csf.mif -datatype bit
dwi2response: Output of failed command:
mrcalc: [ERROR] not enough operands in stack for operation "if"
mrthreshold: [ERROR] no filename supplied to standard input (broken pipe?)
mrthreshold: [ERROR] error opening image "-"
mrcalc: [ERROR] error converting string "-"
dwi2response: Changing back to original directory (/mnt/projects/LifeMabs/DWItest/mrtrix_2000/b2000_50dir)
dwi2response: Script failed while executing the command: mrcalc _crudenonwm.mif safe_sdm.mif  -subtract 0 -if - | mrthreshold - - -mask _crudenonwm.mif | mrcalc _crudenonwm.mif - 0 -if crude_csf.mif -datatype bit
dwi2response: For debugging, inspect contents of temporary directory: /tmp/dwi2response-tmp-46Y7GP/

:sweat_smile:


#14

Hi @Nayo,

Looks to me like this might be a similar problem to what’s going on in this parallel discussion: One of the mask images derived during the response function estimation algorithm is empty, which is resulting in a call to mrstats not returning any result. In your particular case this subsequently results in a malformed call to mrcalc (See the double space between ‘safe_sdm.mif’ and ‘-subtract’? There’s supposed to be a number in the middle), but it’s the preceding empty mask that’s the fundamental problem. My guess is that there’s something particularly unusual about your data that’s breaking one of the assumptions within the dwi2response dhollander algorithm; so the best bet would be to make an example image available so that the problem can be reproduced at our end.

Cheers
Rob


Error using dwi2response msmt_5tt
#15

Hey @Nayo,

Sorry, but it seems I had completely missed your post here! Rob is right in that this is a combination of a specific mask in the algorithm being empty, and then the way how this makes the script crash due to a particular technical caveat in a command that just surfaced for another user in another context too… I’ll eventually add in a useful error message for this particular event. You’d get that error message in that case any way, so you’d still not be able to proceed.

In your particular case, it seems the region within the initial brain mask that has FA values below 0.2 is empty. That would be very odd indeed… Maybe even the original brain masking went wrong. If you want, you could make the DWI image that you provides to the script available somewhere (via Dropbox of something), and let me know in private (you can use the messaging feature on the forum, if you’re ok with it); so I can take a look at it.

Anyway, it seems you’re after single-shell processing here; so for now (while you may share the dataset with me in between), you could still proceed using just the tournier algorithm in dwi2response, combined with dwi2fod csd afterwards, for simple single-shell single-tissue CSD.

Hope this helps a bit (for now). :slight_smile:

Cheers,
Thijs


#16

Hi bjeurissen

I met the same problem using hcp data(the subject 163129,166438,169747), and I checked the 5tt.mif .I think the 5tt.mif is right.Are there other reasons causing the problem ?

Thanks


#17

Hello,
I got a similar problem with my data, the mask is empty. Something that I know is NOT recommendable was performed in order to obtain and embed bval & bvec into dwi.mif file (not sure whether this is the cause of the problem). I got only raw data, converted to nii format by Bru2Nii, and extracted bvec and bval from $PVM_DwDir and $PVM_DwEffBval respectively from a file called “method” at the raw data, and arranged it in fsl format.

My data looks like this:
Carloss-MacBook-Pro:mrtrix_test carlosengutierrez$ mrinfo 14.mif
************************************************
Image: “14.mif”
************************************************
Dimensions: 128 x 30 x 128 x 130
Voxel size: 3.5 x 7 x 3.5 x 4
Data strides: [ 1 3 -2 4 ]
Format: MRtrix
Data type: signed 16 bit integer (little endian)
Intensity scaling: offset = 0, multiplier = 4.3775600000000001e-05
Transform: 1 -0.0005371 0.003784 -212.5
0.00053 1 0.001875 -86.65
-0.003785 -0.001873 1 -217.8
comments: 13_HR_DTI_128axis_b3000
dw_scheme: [ 130 entries ]
mrtrix_version: 005da18a

And the shells:
Carloss-MacBook-Pro:mrtrix_test carlosengutierrez$ mrinfo 14.mif -shells -shellcounts
0 3018.33 
2 128 

Carloss-MacBook-Pro:mrtrix_test carlosengutierrez$ mrinfo 14.mif -dwgrad
         -0           0           0           0
         -0           0           0           0
  -0.140642    0.989951   0.0147479     3041.35
  0.0827185    0.995958  -0.0350076     3042.13
  0.0306206    0.980789    0.192654     3040.53
  -0.375125    0.925683   0.0489144     3037.26
  -0.081265    0.973005   -0.216003     3034.51
   0.144104    0.952087   -0.269749     3031.95
    0.29863    0.950621  -0.0844939     3033.45
    0.24308    0.956728    0.159948     3032.08
 -0.0786406    0.924687    0.372518     3032.94

Then I tried the following:
a) dhollander:
Carloss-MacBook-Pro:mrtrix_test carlosengutierrez$ dwi2response dhollander 14.mif wm.txt gm.txt csf.txt

Command: mrcalc safe_mask.mif safe_fa.mif 0 -if 0.2 -gt crude_wm.mif -datatype bit
Command: mrcalc crude_wm.mif 0 safe_mask.mif -if _crudenonwm.mif -datatype bit
mrstats: [ERROR] Cannot output statistic of interest; no values read (empty mask?)
Command: mrcalc _crudenonwm.mif safe_sdm.mif -subtract 0 -if - | mrthreshold - - -mask _crudenonwm.mif | mrcalc _crudenonwm.mif - 0 -if crude_csf.mif -datatype bit
dwi2response:
dwi2response: [ERROR] Command failed: mrcalc _crudenonwm.mif safe_sdm.mif -subtract 0 -if - | mrthreshold - - -mask _crudenonwm.mif | mrcalc _crudenonwm.mif - 0 -if crude_csf.mif -datatype bit
dwi2response: Output of failed command:
mrcalc: [ERROR] not enough operands in stack for operation "if"
mrthreshold: [ERROR] no filename supplied to standard input (broken pipe?)
mrthreshold: [ERROR] error opening image "-"
mrcalc: [ERROR] error converting string "-"
dwi2response: Changing back to original directory (/Users/carlosengutierrez/datathon/mrtrix_test)
dwi2response: Script failed while executing the command: mrcalc _crudenonwm.mif safe_sdm.mif -subtract 0 -if - | mrthreshold - - -mask _crudenonwm.mif | mrcalc _crudenonwm.mif - 0 -if crude_csf.mif -datatype bit
dwi2response: For debugging, inspect contents of temporary directory: /tmp/dwi2response-tmp-ZWA1YX/

b) tournier

Carloss-MacBook-Pro:mrtrix_test carlosengutierrez$ dwi2response tournier 14.mif 14_tournier.mif -grad encode.b
....
Command: fod2fixel iter0_FOD.mif -peak iter0_peaks.msf -mask mask.mif -fmls_no_thresholds
dwi2response: Deleting file: iter0_FOD.mif
Command: fixel2voxel iter0_peaks.msf split_value iter0_amps.mif
dwi2response: 
dwi2response: [ERROR] Command failed: fixel2voxel iter0_peaks.msf split_value iter0_amps.mif
dwi2response: Output of failed command:
fixel2voxel: [ERROR] fixel image is empty
dwi2response: Changing back to original directory (/Users/carlosengutierrez/datathon/mrtrix_test)
dwi2response: Script failed while executing the command: fixel2voxel iter0_peaks.msf split_value iter0_amps.mif
dwi2response: For debugging, inspect contents of temporary directory: /tmp/dwi2response-tmp-EZILZ5/

Please advice,
Thanks, Carlos.


#18

Hello,
About mask images getting empty, I found the problem for my case, the gradient data was not in MRTrix format.
I used commnad convert_bruker to obtain mih file with the correct gradient data format, then I converted mih to mif.

$ convert_bruker 14/pdata/1/2dseq 14.mih
$ mrconvert 14.mih 14_1.mif

You can see the before-after info of my data, dimensions, voxel size and Transform matrix changed


Image: “14.nii”


Dimensions: 128 x 30 x 128 x 130
Voxel size: 3.5 x 7 x 3.5 x 4
Data strides: [ 1 3 -2 4 ]
Format: NIfTI-1.1
Data type: signed 16 bit integer (little endian)
Intensity scaling: offset = 0, multiplier = 4.3775577069027349e-05
Transform: 1 -0.0005371 0.003784 -212.5
0.00053 1 0.001875 -86.65
-0.003785 -0.001873 1 -217.8
comments: 13_HR_DTI_128axis_b3000


Image: “14_1.mif”


Dimensions: 128 x 128 x 30 x 130
Voxel size: 0.35 x 0.35 x 0.7 x ?
Data strides: [ 1 2 3 4 ]
Format: MRtrix
Data type: signed 16 bit integer (little endian)
Intensity scaling: offset = 0, multiplier = 1
Transform: 1 0 0 -22.23
0 1 0 -22.23
0 0 1 -10.15
dw_scheme: [ 130 entries ]
mrtrix_version: 005da18a

Thanks, Carlos.


#19

Thanks for reporting back yourself already @Carlos_Gutierrez. I was already thinking it must’ve been something as fundamental as that; it definitely didn’t seem specific to (any algorithm for) response function selection.


#20

Hi,

I have a similar issue with a 2D rat invivo dataset.

dwi2response with tournier works and dhollander gives the empty mask error!

However, when I convert the 2D dataset into a cube (matching the 3rd dimension with the other 2) dhollander works!

Karthik