Dwipreproc - eddy ERROR

Apparently the problem for me seems to be that the original dicoms were in mosaic form.

Once I got some nifti files, and after I converted to .mif and added the gradient information there was no more problem.

How does mrtrix3 handle this format, and how can it be converted?

OK, I can see how this could cause some confusion…

MRtrix3 should handle this format just fine - but where it can fail is when the DICOM files have been processed by some third party application, particularly for the purposes of anonymisation - this process often strips out so-called private tags from the DICOM files, which are vendor-specific. The fact that you needed to add the gradient encoding information separately suggests that you would indeed have been provided with a stripped-down version of the original Siemens DICOM, from which the proprietary Siemens CSA tags have been removed. Unfortunately, these tags contain the information required to correctly read the Siemens mosaic format, as well as the diffusion encoding information. More details here if you’re interested.

My guess is if you go back to the original DICOMs, it would work fine. I’m not sure where you got the NIfTI version from, but presumably that was either converted from the original DICOMs, or converted using a more heuristic approach (you can guess what the right parameters would be for the mosaic, up to a certain point). But MRtrix3 currently relies on the Siemens private CSA tags to read these correctly.

If you are able to go back to the original DICOMs and run eddy without a problem, it would be useful for us to get an example of your images. If indeed some anonymisation process is preventing the correct conversion of Siemens DICOMs, this error should preferably be raised well before dwipreproc gets to the point of invoking eddy. I would also be curious to know exactly what type of input image is able to survive mrconvert but cause a segmentation violation in eddy (FMRIB may also be interested to see such an image).

@rsmith: the problem is that the DICOM data were provided in mosaic format. If you strip out the Siemens CSA info, MRtrix3 can’t reformat these into 3 volumes - they remain as 2D slices with all the slices in a single large montage. If the user also provides the DW gradient info manually (which they did), then I guess eddy is the first command that will actually fail (no topup used on this invocation). There’s a good chance the mask produced by dwi2mask will also be empty, given that the cleanup operations operate at a 3D level, and this is likely to cause eddy to then crash out - but dwi2mask probably won’t itself crash (might be a idea for it to raise an error if the mask is empty, though…).

So either an empty mask, or the fact that eddy's been presented with a single slice, which means even trilinear interpolation can’t be done, for example, or there can’t be any through-slice gradients for registration, or any number of other related problems…

Hi, I also encountered the same problem that dwipreproc doesn’t work during eddy command.
I tried to do dwipreproc from DICOM and NIFTI after mrconvert respectively, still have the same error. When I tried to convert DICOM to NIFTI by MRICron and run dwipreproc, it does work.

I found that dimensions of the nifti files from the two different ways are a little different. Actually, this data should be 128 x 128 x 75 x 33. But I’m not sure if this is an important problem for that eddy doesn’t work.

From MRICron
Dimensions: 128 x 128 x 75 x 33
Voxel size: 2 x 2 x 2 x 10.0674
Data strides: [ -1 2 3 4 ]
Format: NIfTI-1.1 (GZip compressed)
Data type: signed 16 bit integer (little endian)
Intensity scaling: offset = 0, multiplier = 2.5892550945281982
Transform: 0.9939 -0.1092 0.01662 -114.9
0.1103 0.988 -0.1081 -124.4
-0.004625 0.1092 0.994 -85.37
comments: TE=91.52799988;sec=73829.0400

From MRtrix
Dimensions: 128 x 128 x 75 x 34
Voxel size: 2 x 2 x 2 x ?
Data strides: [ -1 -2 3 4 ]
Format: NIfTI-1.1 (GZip compressed)
Data type: unsigned 16 bit integer (little endian)
Intensity scaling: offset = 0, multiplier = 2.5892550945281982
Transform: 0.9939 -0.1092 0.01662 -114.9
0.1103 0.988 -0.1081 -124.4
-0.004625 0.1092 0.994 -85.37
mrtrix_version: 3.0_RC2-2-gca446aad

OK, that sounds like a different problem again from the others mentioned in this thread…

My guess is your data come from a Philips scanner…? They have a habit of adding an additional image at the end of the series, which I think is a mean DWI image, computed from the acquired data. Clearly this is not an actual diffusion-weighted image, and typically its DW encoding entry looks ‘odd’ - on our scanners, it shows up as a 0 0 0 1000: a zero vector with a non-zero b-value… This is clearly going cause all manner of confusion for any downstream processing. You could verify this by inspecting the bvecs & bvals files to see if that is the case here (i.e. last column of bvecs contains zeros, but last entry of bvals is non-zero).

Assuming that is indeed the problem, you’ll probably find that updating your installation to the latest version, freshly released a few days ago, fixes the issue since it contains improved handling of DICOM data that should actually be able to deal with precisely these types of data (since the additional computed image shows up with a different ImageType entry in the DICOM headers)…

Thanks for your response
Yes, the data come from a Philips scanner. Their data is as your said.
I build the MRtrix (RC2-2-gca446aad) at Aug 9 2017. I guess that is new version.
I will try again

Hello

I have had a similar issue to the above. When I run dwipreproc -rpe_none DWI_denoise.mif -fslgrad bvecs bvals DWI_eddy.mif -pe_dir AP I get an error,

[ERROR] Command failed: eddy_openmp --imain=dwi.nii --mask=mask.nii --acqp=eddy_config.txt --index=eddy_indices.txt --bvecs=bvecs --bvals=bvals --out=dwi_post_eddy (dwipreproc:467)

If I invoke the failing eddy command in the temporary folder it works. You suggested a buggy python or interference from locale settings. Whilst not an mrtrix issue I was wondering if you knew of how I could go about fixing this? Or given that eddy is running in the temporary folder should I just use the eddy command manually in a script if that’s easier?

Thanks for your help!

Paul

Paul: Can you provide the full error message? The earlier mention in this thread of the locale settings was only a hypothesis, and as far as I can see there was no confirmation from any user that this was indeed the cause of their problem. This thread now has a range of different issues reported, so we really need to see the full details in order to narrow down our suggestions.

Of course.

Command:  mrconvert /home/trackhd/Desktop/TrackStorage/Paul/connect/trackon_noddi/NIFTI_ORIG_All/028502628/DWI_dn.mif /home/trackhd/Desktop/TrackStorage/Paul/connect/trackon_noddi/NIFTI_ORIG_All/028502628/dwipreproc-tmp-VBJUWX/dwi.mif -fslgrad /home/trackhd/Desktop/TrackStorage/Paul/connect/trackon_noddi/NIFTI_ORIG_All/028502628/bvecs /home/trackhd/Desktop/TrackStorage/Paul/connect/trackon_noddi/NIFTI_ORIG_All/028502628/bvals
dwipreproc: Changing to temporary directory (/home/trackhd/Desktop/TrackStorage/Paul/connect/trackon_noddi/NIFTI_ORIG_All/028502628/dwipreproc-tmp-VBJUWX/)
dwipreproc: [WARNING] Total readout time not provided at command-line; assuming sane default of 0.1
Command:  mrinfo dwi.mif -export_grad_mrtrix grad.b
Command:  dwi2mask dwi.mif - | maskfilter - dilate - | mrconvert - mask.nii -datatype float32 -stride -1,+2,+3
Command:  mrconvert dwi.mif -import_pe_table dwi_manual_pe_scheme.txt dwi.nii -stride -1,+2,+3,+4 -export_grad_fsl bvecs bvals -export_pe_eddy eddy_config.txt eddy_indices.txt
Command:  eddy_openmp --imain=dwi.nii --mask=mask.nii --acqp=eddy_config.txt --index=eddy_indices.txt --bvecs=bvecs --bvals=bvals --out=dwi_post_eddy
dwipreproc: 
dwipreproc: **[ERROR]** Command failed: eddy_openmp --imain=dwi.nii --mask=mask.nii --acqp=eddy_config.txt --index=eddy_indices.txt --bvecs=bvecs --bvals=bvals --out=dwi_post_eddy (dwipreproc:467)
dwipreproc: Output of failed command:
I'm thrown
terminate called after throwing an instance of 'EDDY::KMatrixException'
  what():  KMatrixException: msg=MultiShellKMatrix::SetDiffusionPar: Data not shelled
dwipreproc: Changing back to original directory (/home/trackhd/Desktop/TrackStorage/Paul/connect/trackon_noddi/NIFTI_ORIG_All/028502628)
dwipreproc: Script failed while executing the command: eddy_openmp --imain=dwi.nii --mask=mask.nii --acqp=eddy_config.txt --index=eddy_indices.txt --bvecs=bvecs --bvals=bvals --out=dwi_post_eddy
dwipreproc: For debugging, inspect contents of temporary directory: /home/trackhd/Desktop/TrackStorage/Paul/connect/trackon_noddi/NIFTI_ORIG_All/028502628/dwipreproc-tmp-VBJUWX/

Then by invoking eddy manually in the temp folder like so; eddy --imain=dwi.nii --mask=mask.nii --acqp=eddy_config.txt --index=eddy_indices.txt --bvecs=bvecs --bvals=bvals --out=dwi_eddy it works to produce an eddy corrected file.

Thanks for your help

The actual error is here:

Your b-values are not sufficiently consistent for each shell, and eddy throws out the data in case they were not acquired using a multi-shell scheme. If you know your data are multi-shell (not DSI), you can force eddy to accept your data with the right option - see the FSL wiki for details. You should be able to pass that option using the -eddy_options option to dwipreproc*.

As to why your manual eddy call works, I’m guessing this is because your eddy and eddy_openmp are from different versions of FSL, and one of them is more strict than the other. On my system, FSL 5.0.8 only provides eddy, while 5.0.9 provides eddy_openmp and eddy_cuda - no eddy as such…

*superseded by dwifslpreproc in version 3.0.0

One other possibility of what might be going on here, since I’ve seen a couple of people being forced to use the --data_is_shelled option:

In MRtrix3, we run a clustering algorithm in order to group volumes into shells according to b-value, which accounts for the fact that the b-values of those volumes may not be precisely equivalent. In addition to the b-values that are provided in the DICOMs, these values may additionally be perturbed by MRtrix3 when it normalises each diffusion gradient direction and applies the corresponding scaling to the b-value. Therefore, if eddy does not account for the possibility of small variations in b-value, it’s possible that a bvecs / bvals pair created by an MRtrix3 command (e.g. in the middle of dwipreproc) may cause eddy to stumble, whereas using the “original” bvecs / bvals and invoking eddy directly may work.

Just a hypothesis right now… Wouldn’t explain why @uclpz was able to run eddy directly from within the script temporary directory, that’s more likely an eddy version mismatch as @jdtournier suggested. But just putting the idea out there in case many people are encountering this “Data not shelled” issue.

Yeah, depending on your FSL version, the definition of “shelled” (or not) may be rather strict. This discussion was historically started somewhere here, if I recall correctly: https://www.jiscmail.ac.uk/cgi-bin/webadmin?A2=fsl;9bfdeaed.1510

Note that there’s also criteria on numbers of directions per shell, and especially these used to be (or are still) rather strict, resulting in rejection of even quite “common” protocols. I’m not sure what their exact constraints/heuristics currently are, or whether these are explicitly documented somewhere though…

Thanks for your help on this. My data is definitely multishell. I did inspect my bvalues which are consistent for each shell - 0, 700, 2000. Curiously if I use the data is shelled option using the command;

dwipreproc DWI_dn.mif DWI_eddy_bval.mif -rpe_none -pe_dir AP -fslgrad bvecs bvals -eddy_options "--data_is_shelled " 

I now get the following error below. Any ideas to the cause of this? Thanks again for all your help.

Traceback (most recent call last):
  File "/midas-data/software/mrtrix3_25.5.17/bin/dwipreproc", line 506, in <module>
    run.command('mrconvert dwi_post_eddy' + fsl_suffix + ' result.mif' + stride_option + ' -fslgrad ' + bvecs_path + ' bvals')
  File "/midas-data/software/mrtrix3_25.5.17/lib/mrtrix3/run.py", line 61, in command
    shebang = _shebang(line[0])
  File "/midas-data/software/mrtrix3_25.5.17/lib/mrtrix3/run.py", line 329, in _shebang
    path = find_executable(exeName(item))
  File "/usr/lib64/python2.6/distutils/spawn.py", line 191, in find_executable
    if not os.path.isfile(executable):
  File "/usr/lib64/python2.6/genericpath.py", line 29, in isfile
    st = os.stat(path)
TypeError: stat() argument 1 must be encoded string without NULL bytes, not str
[trackhd@pc8-024 060443961]$ dwipreproc DWI_dn.mif DWI_eddy_bval.mif -rpe_none -pe_dir AP -fslgrad bvecs bvals -eddy_options "--data_is_shelled "

I think you have to add one whitespace character after your double quotes, i.e.
" –data_is_shelled "

Given this output:

It seems to me more likely that the problem might be due to using python version 2.6. As stated in the docs, we require at least version 2.7, at least for the configure & build steps… Worth trying with version 2.7 or 3.X, it might be as simple as that.

Hello all,

I am having a similar issue with the ones mentioned above. When running " dwipreproc data_denoise.mif/ dwi_preprocessed.mif -rpe_none -pe_dir j- " I get the following error:

[ERROR] Command failed: eddy_openmp --imain=dwi.nii --mask=mask.nii --acqp=eddy_config.txt --index=eddy_indices.txt --bvecs=bvecs --bvals=bvals --out=dwi_post_eddy (dwipreproc:518)
dwipreproc: Output of failed command:
eddy: msg=AcqPara::AcqPara: Unrealistic read-out time
terminate called after throwing an instance of 'EDDY::EddyException'
  what():  eddy: msg=AcqPara::AcqPara: Unrealistic read-out time
dwipreproc: Changing back to original directory (/home/suminiky/Downloads/MRTRIX)
dwipreproc: Script failed while executing the command: eddy_openmp --imain=dwi.nii --mask=mask.nii --acqp=eddy_config.txt --index=eddy_indices.txt --bvecs=bvecs --bvals=bvals --out=dwi_post_eddy
dwipreproc: For debugging, inspect contents of temporary directory: /home/suminiky/Downloads/MRTRIX/dwipreproc-tmp-Y06916/

Afterwards, when running eddy in the temp folder I get this:

eddy: msg=AcqPara::AcqPara: Unrealistic read-out time
terminate called after throwing an instance of 'EDDY::EddyException'
  what():  eddy: msg=AcqPara::AcqPara: Unrealistic read-out time
Aborted (core dumped)

Any ideas on what the issue could be?

Best,
Sabina

Hi Sabina,

This is in fact an unrelated error, which has been reported here. This is not an error within dwipreproc, but something that is generated within eddy that dwipreproc is simply passing along. It claims that the EPI total readout time of your data is not “realistic”, and hence there must have been some kind of error made during your data preparation.

Due to the relationship between readout time, field inhomogeneity estimation, and geometric correction of EPI distortions, you should simply be able to e.g. halve the readout times in all of your images, and processing should operate as expected. The mrconvert -set_property option should be useful for this, in addition to this development-branch documentation page (not yet part of the main documentation, but will be soon).

Rob

Hi Rob,

The answer you gave me worked last time, so thank you! However, now I need to analyze a new set of data and I am yet again receiving an error, but a different one:

dwipreproc: [ERROR] Command failed: eddy_openmp --imain=dwi.nii --mask=mask.nii --acqp=eddy_config.txt --index=eddy_indices.txt --bvecs=bvecs --bvals=bvals --out=dwi_post_eddy (dwipreproc:518)
dwipreproc: Output of failed command:
I’m thrown
terminate called after throwing an instance of ‘EDDY::KMatrixException’
what():
dwipreproc: Changing back to original directory (/home/suminiky/Documents/DATA/Menzel_preop)
dwipreproc: Script failed while executing the command: eddy_openmp --imain=dwi.nii --mask=mask.nii --acqp=eddy_config.txt --index=eddy_indices.txt --bvecs=bvecs --bvals=bvals --out=dwi_post_eddy

What could be wrong this time?

Best,
Sabina

Hello all,

I am having a similar issue with the ones mentioned above. When running “dwipreproc dwi_denoised.mif dwi_processed.mif -pe_dir AP -rpe_all -readout_time 0.124” I get the following error:

Command: eddy_openmp --imain=dwi.nii --mask=mask.nii --acqp=eddy_config.txt --index=eddy_indices.txt --bvecs=bvecs --bvals=bvals --topup=field --out=dwi_post_eddy
dwipreproc:
dwipreproc: [ERROR] Command failed: eddy_openmp --imain=dwi.nii --mask=mask.nii --acqp=eddy_config.txt --index=eddy_indices.txt --bvecs=bvecs --bvals=bvals --topup=field --out=dwi_post_eddy (dwipreproc:518)
dwipreproc: Output of failed command:
dwipreproc: Changing back to original directory (/home/wpy/Desktop/FBA/p01)
dwipreproc: Script failed while executing the command: eddy_openmp --imain=dwi.nii --mask=mask.nii --acqp=eddy_config.txt --index=eddy_indices.txt --bvecs=bvecs --bvals=bvals --topup=field --out=dwi_post_eddy
dwipreproc: For debugging, inspect contents of temporary directory: /home/wpy/Desktop/FBA/p01/dwipreproc-tmp-L9W43S/

Do you have any ideas of what could be wrong?

Regards,
Pinyi Wang