I have a problem using dwifslpreproc from the latest MRtrix version installed via conda on Linux Ubuntu 18.04.
I have dwi data with AP and PA phase encoding of all 64 diffusion directions and a single diffusion gradient strength (bval = 1000).
The files have 65 volumes (1 b0 volume + 64 directions)
The data is, converted from nifti to mif (mrconvert, adding the diffusion scheme bvec and bval files), denoised (dwidenoise) and gibbs ringing is removed (mrdegibbs).
Than the 2 dwi files (AP and PA) are concatenated,
Due to me trying to be increasingly clever with the handling of image header key-value information in scripts, dwifslpreproc is currently erroneously discarding changes to the diffusion gradient table that occur during script execution. This is obviously most noticeable when using -rpe_all (or -rpe_header when explicit volume recombination occurs), as you end up with a gradient table with twice as many rows as it should have; but it also means that the motion-corrected bvecs table provided by eddy is currently not propagating to the output file.
Thanks for the report; I’ll push the fix to this and clean up a few other speed humps and hopefully we’ll push out 3.0.1 soon. In the meantime if you need to be able to proceed with your processing, this is the only change that you need to make to your local dwifslpreproc script file.
I edited the dwifslpreproc script file on my Macbook pro as per your indication:
At least on the Macbook Pro it is producing errors. It finishes the eddyqc report but doest not output the final geomcorr.mif file, Just to be sure, I then changed the dwifslpreproc script back (i.e commenting out your edits) and it finished without error. I’ve copied the terminal output of both versions below. geomcorr_eddyqc3.mif should have been the output of your corrected dwifslpreproc script. geomcorr_eddyqc4.mif is the output of the script version that came with the MRtrix3 3.0.0 download:
The following command (that fails with the edited script) had finished previously without error:
% dwifslpreproc degibbs.mif geomcorr_eddyqc3.mif -pe_dir pa -readout_time 0.0581166 -rpe_none -eddy_options " --repol" -eddyqc_text directory.eddy_qc3 -nocleanup
RERUN after R Smith's fix of code May 31, 2020
% dwifslpreproc degibbs.mif geomcorr_eddyqc3.mif -pe_dir pa -readout_time 0.0581166 -rpe_none -eddy_options " --repol" -eddyqc_text directory.eddy_qc3 -nocleanup -force
dwifslpreproc:
dwifslpreproc: Note that this script makes use of commands / algorithms that have relevant articles for citation; INCLUDING FROM EXTERNAL SOFTWARE PACKAGES. Please consult the help page (-help option) for more information.
dwifslpreproc:
dwifslpreproc: [WARNING] Output file 'geomcorr_eddyqc3.mif' already exists; will be overwritten at script completion
dwifslpreproc: [WARNING] Output eddy QC directory already contains relevant files; these will be overwritten on completion
dwifslpreproc: Generated scratch directory: /Users/jameshartzell/Documents/MRtrix3/S6_test/2nd.test.course.steps/dwifslpreproc-tmp-MAS2V8/
Command: mrconvert /Users/jameshartzell/Documents/MRtrix3/S6_test/2nd.test.course.steps/degibbs.mif /Users/jameshartzell/Documents/MRtrix3/S6_test/2nd.test.course.steps/dwifslpreproc-tmp-MAS2V8/dwi.mif -json_export /Users/jameshartzell/Documents/MRtrix3/S6_test/2nd.test.course.steps/dwifslpreproc-tmp-MAS2V8/dwi.json
dwifslpreproc: Changing to scratch directory (/Users/jameshartzell/Documents/MRtrix3/S6_test/2nd.test.course.steps/dwifslpreproc-tmp-MAS2V8/)
Command: dirstat dwi.mif -output asym
Command: mrinfo dwi.mif -export_grad_mrtrix grad.b
Command: dwi2mask dwi.mif - | maskfilter - dilate - | mrconvert - eddy_mask.nii -datatype float32 -strides -1,+2,+3
Command: mrconvert dwi.mif -import_pe_table dwi_manual_pe_scheme.txt eddy_in.nii -strides -1,+2,+3,+4 -export_grad_fsl bvecs bvals -export_pe_eddy eddy_config.txt eddy_indices.txt
Command: eddy_cuda7.5 --imain=eddy_in.nii --mask=eddy_mask.nii --acqp=eddy_config.txt --index=eddy_indices.txt --bvecs=bvecs --bvals=bvals --repol --out=dwi_post_eddy --verbose
dwifslpreproc: CUDA version of 'eddy' was not successful; attempting OpenMP version
Command: eddy --imain=eddy_in.nii --mask=eddy_mask.nii --acqp=eddy_config.txt --index=eddy_indices.txt --bvecs=bvecs --bvals=bvals --repol --out=dwi_post_eddy --verbose
Command: eddy_quad dwi_post_eddy -idx eddy_indices.txt -par eddy_config.txt -m eddy_mask.nii -b bvals
Command: mrconvert dwi_post_eddy.nii.gz result.mif -strides -1,-2,3,4 -fslgrad dwi_post_eddy.eddy_rotated_bvecs bvals
Function: shutil.copy('dwi_post_eddy.eddy_parameters', '/Users/jameshartzell/Documents/MRtrix3/S6_test/2nd.test.course.steps/directory.eddy_qc3/eddy_parameters')
Function: shutil.copy('dwi_post_eddy.eddy_movement_rms', '/Users/jameshartzell/Documents/MRtrix3/S6_test/2nd.test.course.steps/directory.eddy_qc3/eddy_movement_rms')
Function: shutil.copy('dwi_post_eddy.eddy_restricted_movement_rms', '/Users/jameshartzell/Documents/MRtrix3/S6_test/2nd.test.course.steps/directory.eddy_qc3/eddy_restricted_movement_rms')
Function: shutil.copy('dwi_post_eddy.eddy_post_eddy_shell_alignment_parameters', '/Users/jameshartzell/Documents/MRtrix3/S6_test/2nd.test.course.steps/directory.eddy_qc3/eddy_post_eddy_shell_alignment_parameters')
Function: shutil.copy('dwi_post_eddy.eddy_post_eddy_shell_PE_translation_parameters', '/Users/jameshartzell/Documents/MRtrix3/S6_test/2nd.test.course.steps/directory.eddy_qc3/eddy_post_eddy_shell_PE_translation_parameters')
Function: shutil.copy('dwi_post_eddy.eddy_outlier_report', '/Users/jameshartzell/Documents/MRtrix3/S6_test/2nd.test.course.steps/directory.eddy_qc3/eddy_outlier_report')
Function: shutil.copy('dwi_post_eddy.eddy_outlier_map', '/Users/jameshartzell/Documents/MRtrix3/S6_test/2nd.test.course.steps/directory.eddy_qc3/eddy_outlier_map')
Function: shutil.copy('dwi_post_eddy.eddy_outlier_n_stdev_map', '/Users/jameshartzell/Documents/MRtrix3/S6_test/2nd.test.course.steps/directory.eddy_qc3/eddy_outlier_n_stdev_map')
Function: shutil.copy('dwi_post_eddy.eddy_outlier_n_sqr_stdev_map', '/Users/jameshartzell/Documents/MRtrix3/S6_test/2nd.test.course.steps/directory.eddy_qc3/eddy_outlier_n_sqr_stdev_map')
dwifslpreproc: [ERROR] Unhandled Python exception:
dwifslpreproc: [ERROR] TypeError: Invalid input to run.function()
dwifslpreproc: [ERROR] Traceback:
dwifslpreproc: [ERROR] /usr/local/bin/dwifslpreproc:1172 (in execute())
dwifslpreproc: [ERROR] run.function(shutil.rmtree(os.path.join(eddyqc_path, 'quad')))
dwifslpreproc: [ERROR] /usr/local/mrtrix3/lib/mrtrix3/run.py:472 (in function())
dwifslpreproc: [ERROR] raise TypeError('Invalid input to run.function()')
dwifslpreproc: Scratch directory retained; location: /Users/jameshartzell/Documents/MRtrix3/S6_test/2nd.test.course.steps/dwifslpreproc-tmp-MAS2V8/
(base) jameshartzell@Jamess-MacBook-Pro 2nd.test.course.steps %
RERUN AFTER CHANGING THE dwifslpreproc SCRIPT BACK TO THE WAY IT WAS:
(base) jameshartzell@Jamess-MacBook-Pro 2nd.test.course.steps % dwifslpreproc degibbs.mif geomcorr_eddyqc4.mif -pe_dir pa -readout_time 0.0581166 -rpe_none -eddy_options " --repol" -eddyqc_text directory.eddy_qc4 -nocleanup -force
dwifslpreproc:
dwifslpreproc: Note that this script makes use of commands / algorithms that have relevant articles for citation; INCLUDING FROM EXTERNAL SOFTWARE PACKAGES. Please consult the help page (-help option) for more information.
dwifslpreproc:
dwifslpreproc: [WARNING] Output file 'geomcorr_eddyqc4.mif' already exists; will be overwritten at script completion
dwifslpreproc: [WARNING] Output eddy QC directory already contains relevant files; these will be overwritten on completion
dwifslpreproc: Generated scratch directory: /Users/jameshartzell/Documents/MRtrix3/S6_test/2nd.test.course.steps/dwifslpreproc-tmp-T8RHSS/
Command: mrconvert /Users/jameshartzell/Documents/MRtrix3/S6_test/2nd.test.course.steps/degibbs.mif /Users/jameshartzell/Documents/MRtrix3/S6_test/2nd.test.course.steps/dwifslpreproc-tmp-T8RHSS/dwi.mif -json_export /Users/jameshartzell/Documents/MRtrix3/S6_test/2nd.test.course.steps/dwifslpreproc-tmp-T8RHSS/dwi.json
dwifslpreproc: Changing to scratch directory (/Users/jameshartzell/Documents/MRtrix3/S6_test/2nd.test.course.steps/dwifslpreproc-tmp-T8RHSS/)
Command: dirstat dwi.mif -output asym
Command: mrinfo dwi.mif -export_grad_mrtrix grad.b
Command: dwi2mask dwi.mif - | maskfilter - dilate - | mrconvert - eddy_mask.nii -datatype float32 -strides -1,+2,+3
Command: mrconvert dwi.mif -import_pe_table dwi_manual_pe_scheme.txt eddy_in.nii -strides -1,+2,+3,+4 -export_grad_fsl bvecs bvals -export_pe_eddy eddy_config.txt eddy_indices.txt
Command: eddy_cuda7.5 --imain=eddy_in.nii --mask=eddy_mask.nii --acqp=eddy_config.txt --index=eddy_indices.txt --bvecs=bvecs --bvals=bvals --repol --out=dwi_post_eddy --verbose
dwifslpreproc: CUDA version of 'eddy' was not successful; attempting OpenMP version
Command: eddy --imain=eddy_in.nii --mask=eddy_mask.nii --acqp=eddy_config.txt --index=eddy_indices.txt --bvecs=bvecs --bvals=bvals --repol --out=dwi_post_eddy --verbose
[Command: eddy_quad dwi_post_eddy -idx eddy_indices.txt -par eddy_config.txt -m eddy_mask.nii -b bvals
Command: mrconvert dwi_post_eddy.nii.gz result.mif -strides -1,-2,3,4 -fslgrad dwi_post_eddy.eddy_rotated_bvecs bvals
Function: shutil.copy('dwi_post_eddy.eddy_parameters', '/Users/jameshartzell/Documents/MRtrix3/S6_test/2nd.test.course.steps/directory.eddy_qc4/eddy_parameters')
Function: shutil.copy('dwi_post_eddy.eddy_movement_rms', '/Users/jameshartzell/Documents/MRtrix3/S6_test/2nd.test.course.steps/directory.eddy_qc4/eddy_movement_rms')
Function: shutil.copy('dwi_post_eddy.eddy_restricted_movement_rms', '/Users/jameshartzell/Documents/MRtrix3/S6_test/2nd.test.course.steps/directory.eddy_qc4/eddy_restricted_movement_rms')
Function: shutil.copy('dwi_post_eddy.eddy_post_eddy_shell_alignment_parameters', '/Users/jameshartzell/Documents/MRtrix3/S6_test/2nd.test.course.steps/directory.eddy_qc4/eddy_post_eddy_shell_alignment_parameters')
Function: shutil.copy('dwi_post_eddy.eddy_post_eddy_shell_PE_translation_parameters', '/Users/jameshartzell/Documents/MRtrix3/S6_test/2nd.test.course.steps/directory.eddy_qc4/eddy_post_eddy_shell_PE_translation_parameters')
Function: shutil.copy('dwi_post_eddy.eddy_outlier_report', '/Users/jameshartzell/Documents/MRtrix3/S6_test/2nd.test.course.steps/directory.eddy_qc4/eddy_outlier_report')
Function: shutil.copy('dwi_post_eddy.eddy_outlier_map', '/Users/jameshartzell/Documents/MRtrix3/S6_test/2nd.test.course.steps/directory.eddy_qc4/eddy_outlier_map')
Function: shutil.copy('dwi_post_eddy.eddy_outlier_n_stdev_map', '/Users/jameshartzell/Documents/MRtrix3/S6_test/2nd.test.course.steps/directory.eddy_qc4/eddy_outlier_n_stdev_map')
Function: shutil.copy('dwi_post_eddy.eddy_outlier_n_sqr_stdev_map', '/Users/jameshartzell/Documents/MRtrix3/S6_test/2nd.test.course.steps/directory.eddy_qc4/eddy_outlier_n_sqr_stdev_map')
Function: shutil.copytree('dwi_post_eddy.qc', '/Users/jameshartzell/Documents/MRtrix3/S6_test/2nd.test.course.steps/directory.eddy_qc4/quad')
Command: mrconvert result.mif /Users/jameshartzell/Documents/MRtrix3/S6_test/2nd.test.course.steps/geomcorr_eddyqc4.mif
dwifslpreproc: Changing back to original directory (/Users/jameshartzell/Documents/MRtrix3/S6_test/2nd.test.course.steps)
dwifslpreproc: Scratch directory retained; location: /Users/jameshartzell/Documents/MRtrix3/S6_test/2nd.test.course.steps/dwifslpreproc-tmp-T8RHSS/
That’s a separate issue; I don’t think the triggering of that problem is related to whether or not you manually fix the prior error, but rather whether or not the script is attempting to over-write an existing output directory. Fix is coming; code is here.
I ran the revised code (i.e. the revised dwifslpeproc with the first fix at 1192 and the second fix at 1172 ) and it finished without error and produced all the correct files.
