Dwifslpreproc error — somewhere after the topup process

hi there,

i’ve encountered an error somewhere after the topup procedure in dwifslpreproc

i ran this:

dwifslpreproc gdntensors.mif DWI_out.mif -rpe_pair -se_epi fmap.nii.gz -pe_dir ap -readout_time 0.12705 -align_seepi -fslgrad CHIF002_ADNI3_advanced_MB_DTI_.bvec CHIF002_ADNI3_advanced_MB_DTI_.bval -eddy_options "--slm=linear"

here are the details from the log.txt

mrconvert /scratch/junhong.yu/CHIF/dicom/CHIF002/DTI/gdntensors.mif /scratch/junhong.yu/CHIF/dicom/CHIF002/DTI/dwifslpreproc-tmp-WQ2SUJ/dwi.mif -fslgrad /scratch/junhong.yu/CHIF/dicom/CHIF002/DTI/CHIF002_ADNI3_advanced_MB_DTI_.bvec /scratch/junhong.yu/CHIF/dicom/CHIF002/DTI/CHIF002_ADNI3_advanced_MB_DTI_.bval -json_export /scratch/junhong.yu/CHIF/dicom/CHIF002/DTI/dwifslpreproc-tmp-WQ2SUJ/dwi.json
mrconvert /scratch/junhong.yu/CHIF/dicom/CHIF002/DTI/fmap.nii.gz /scratch/junhong.yu/CHIF/dicom/CHIF002/DTI/dwifslpreproc-tmp-WQ2SUJ/se_epi.mif
mrconvert dwi.mif dwi_first_bzero.mif -coord 3 0 -axes 0,1,2
mrconvert se_epi.mif - -coord 3 1 | mrcat dwi_first_bzero.mif - se_epi_firstdwibzero.mif -axis 3
mrinfo dwi.mif -export_grad_mrtrix grad.b
mrconvert se_epi_firstdwibzero.mif -coord 2 80 - | mrcat se_epi_firstdwibzero.mif - se_epi_firstdwibzero_pad2.mif -axis 2
mrconvert dwi.mif -coord 2 80 - | mrcat dwi.mif - dwi_pad2.mif -axis 2
mrconvert se_epi_firstdwibzero_pad2.mif topup_in.nii -import_pe_table se_epi_manual_pe_scheme.txt -strides -1,+2,+3,+4 -export_pe_table topup_datain.txt
topup --imain=topup_in.nii --datain=topup_datain.txt --out=field --fout=field_map.nii.gz --config=/usr/local/fsl-6.0.5/etc/flirtsch/b02b0.cnf --verbose
mrconvert dwi_pad2.mif -import_pe_table dwi_manual_pe_scheme.txt - | mrinfo - -export_pe_eddy applytopup_config.txt applytopup_indices.txt
mrconvert dwi_pad2.mif dwi_pad2_pe_0.nii -coord 3 0,1,2,3,4,5,6,7,8,9,10,11,12,13,14,15,16,17,18,19,20,21,22,23,24,25,26,27,28,29,30,31,32,33,34,35,36,37,38,39,40,41,42,43,44,45,46,47,48,49,50,51,52,53,54,55,56,57,58,59,60,61,62,63,64,65,66,67,68,69,70,71,72,73,74,75,76,77,78,79,80,81,82,83,84,85,86,87,88,89,90,91,92,93,94,95,96,97,98,99,100,101,102,103,104,105,106,107,108,109,110,111,112,113,114,115,116,117,118,119,120,121,122,123,124,125,126 -strides -1,+2,+3,+4 -json_export dwi_pad2_pe_0.json
applytopup --imain=dwi_pad2_pe_0.nii --datain=applytopup_config.txt --inindex=1 --topup=field --out=dwi_pad2_pe_0_applytopup.nii --method=jac
mrconvert dwi_pad2_pe_0_applytopup.nii.gz dwi_pad2_pe_0_applytopup.mif -json_import dwi_pad2_pe_0.json

and from the error.txt

dwi2mask dwi_pad2_pe_0_applytopup.mif - | maskfilter - dilate - | mrconvert - eddy_mask.nii -datatype float32 -strides -1,+2,+3

dwi2mask: preloading data for "dwi_pad2_pe_0_applytopup.mif"... [==================================================]
dwi2mask: [ERROR] no valid diffusion gradient table found
dwi2mask: [ERROR] error importing diffusion gradient table for image "dwi_pad2_pe_0_applytopup.mif"
maskfilter: [ERROR] no filename supplied to standard input (broken pipe?)
maskfilter: [ERROR] error opening image "-"
mrconvert: [ERROR] no filename supplied to standard input (broken pipe?)
mrconvert: [ERROR] error opening image "-"

1 Like

so i edited dwifslpreproc by adding -grad grad.b to line 864 and 915 (discovered that there are actually 2 instances of this issue), and this eliminated the ‘no valid diffusion table issue’.

line 864:

run.command('dwi2mask ' + dwi_path + ' - | maskfilter - dilate - | mrconvert - eddy_mask.nii -datatype float32 -strides -1,+2,+3')

line 915:

run.command('mrconvert ' + dwi_path + import_dwi_pe_table_option + dwi_permvols_preeddy_option + ' eddy_in.nii -grad grad.b -strides -1,+2,+3,+4 -export_grad_fsl bvecs bvals -export_pe_eddy eddy_config.txt eddy_indices.txt')

however i encountered another error subsequently

Command:  eddy_openmp --imain=eddy_in.nii --mask=eddy_mask.nii --acqp=eddy_config.txt --index=eddy_indices.txt --bvecs=bvecs --bvals=bvals --topup=field “–slm=linear” --out=dwi_post_eddy --verbose

dwifslpreproc: [ERROR] Unhandled Python exception:
dwifslpreproc: [ERROR]   UnicodeEncodeError: 'ascii' codec can't encode characters in position 3419-3420: ordinal not in range(128)
dwifslpreproc: [ERROR] Traceback:
dwifslpreproc: [ERROR]   /home/junhong.yu/.conda/envs/mrtrix3env/bin/dwifslpreproc:940 (in execute())
dwifslpreproc: [ERROR]     eddy_output_file.write(str(exception_openmp).encode('utf-8', errors='replace'))

I had a look into your issue, and I think the problem is introduced earlier, at this point when we pad the image to have an even number of slices (otherwise topup will fail). The output image of that step is supposed to contain the DW encoding within its header, but it looks like that might not be the case for you.

We rely on the mrcat call to preserve the DW encoding table (which it should), but in some circumstances there are minor differences in precision that can be introduced by the way we handle certain aspects of b-value scaling, and I think this gets detected as a difference in the entries, resulting in the DW scheme information being labelled as ‘variable’ in the output image. We’ll need to investigate how to fix this in the next release…

In your case, it might be sufficient to strip the DW encoding information from the slice to be padded so that the subsequent call to mrcat doesn’t detect a mismatch, by adding -clear dw_scheme to the preceding mrconvert call, e.g. replace that line with:

run.command('mrconvert ' + dwi_path + ' -coord ' + str(axis) + ' ' + str(axis_size-1) + ' -clear dw_scheme - | mrcat ' + dwi_path + ' - ' + new_dwi_path + ' -axis ' + str(axis))

Hopefully that’ll fix it for you… Please give it a shot and let us know how it goes! If it works we’ll implement the fix on our side.


[NOTE: edited to add missing space mentioned in subsequent post]

i got an error with the replaced line

dwifslpreproc: [ERROR] mrconvert dwi.mif -coord 2 80-clear dw_scheme - | mrcat dwi.mif - dwi_pad2.mif -axis 2 (dwifslpreproc:771)
dwifslpreproc: [ERROR] Information from failed command:
dwifslpreproc:
               mrconvert: [ERROR] Expected exactly 2 arguments (3 supplied)
               mrconvert: [ERROR] Usage: mrconvert input output
               mrconvert: [ERROR] Yours: mrconvert dwi.mif dw_scheme -
               mrcat: [ERROR] no filename supplied to standard input (broken pipe?)
               mrcat: [ERROR] error opening image "-"

Sorry, my mistake, I left out an important space. This should be:

... + ' -clear dw_scheme - | mrcat ' + ...

Note the extra space before -clear.

Yes, that resolves the error. However, eddy_openmp seems to run forever—i’m using 16 threads for 4 hours and it has not completed yet. But i understand this is an fsl issue.

1 Like

To provide another report, I ran into the exact same error as @GerardYu last week while processing a new dataset.

dwi2mask: preloading data for "dwi_pad2_pe_0_applytopup.mif"... [==================================================]
dwi2mask: [ERROR] no valid diffusion gradient table found
dwi2mask: [ERROR] error importing diffusion gradient table for image "dwi_pad2_pe_0_applytopup.mif"

Glad I checked here. The edit suggested by @jdtournier got me through that step, and dwifslpreproc completed as normal.

1 Like

Hi there, I wanted to report the same issue:

dwifslpreproc: [ERROR] dwi2mask dwi_pad2_pe_0_applytopup.mif - | maskfilter - dilate - | mrconvert - eddy_mask.nii -datatype float32 -strides -1,+2,+3 (dwifslpreproc:852)
dwifslpreproc: [ERROR] Information from failed command:
dwifslpreproc:
dwi2mask: preloading data for “dwi_pad2_pe_0_applytopup.mif”… [==================================================]
dwi2mask: [ERROR] no valid diffusion gradient table found
dwi2mask: [ERROR] error importing diffusion gradient table for image “dwi_pad2_pe_0_applytopup.mif”
maskfilter: [ERROR] no filename supplied to standard input (broken pipe?)
maskfilter: [ERROR] error opening image “-”
mrconvert: [ERROR] no filename supplied to standard input (broken pipe?)
mrconvert: [ERROR] error opening image “-”

My volume number is 109. However, I won’t be able to edit the actual script because the MRtrix is a shared module on the cluster I am working on. I wonder if there’s any other workaround with this.

Thank you!
Meishan

Hello I have encountered the same issue and haven’t understood clearly which line to I have to replace with:
run.command('mrconvert ' + dwi_path + ' -coord ' + str(axis) + ' ' + str(axis_size-1) + ' -clear dw_scheme - | mrcat ' + dwi_path + ' - ' + new_dwi_path + ' -axis ' + str(axis))

I f you could clarify it I would be glad.
thanks
Rosella

Hi Meishan,

I am facing the same issue, and as you reported I can’t edit the script because of the same. I wonder how you managed to fix it.

Thanks a lot!

Antonio

Hi Antonio,

I ended up contacting our cluster manager and editing the script, and they saved a version for me (so it won’t interfere with the original shared version).

Regards,
Meishan