Hi Derek
Never mind my question.
Apparently, eddy_cuda binary is included in fsl 5.0.11 and I didn’t know that I had to call -cuda explicitly in the dwipreproc command (I thought it will find it automatically!). 
So I did that and my 102d multishell topup+eddy_openmp (8+35 minutes) runtime reduced to (8+15 minutes)! Nice! 
Cheers,
Hamed
Hi Rob,
Your sensible explanations are once again helping keep me sane!
I have tried using the -continue option as you suggested. This appears to get around the msg=MultiBandGroups: Unable to open file error but still returns UnicodeDecodeError: ‘utf8’ codec can’t decode byte 0x9e in position 1: invalid start byte
Therefore, I think adding the full path to the slspec.txt file for the --slspec flag might have worked, however the file “my_slspec.txt” or “myslspec.txt” is failing to be read regardless.
From reviewing Hamed’s recent post and the corresponding GitHub issue a solution did not jump out at me. Did I miss something?
It seems like the update for this should hopefully be available soon (days not weeks) 
so I’m thinking my best option may be to wait it out. Although, this may not solve the unicode issue…
But do you think this would be an option for Philips DICOMs as well? I guess they don’t include this information. Do they?
Correct. From the data I have at hand, Philips never include this information. Conversely, GE provide the acquisition time of each slice as being identical to the time of the commencement of the acquisition. In both cases MRtrix3 will detect that there is no timing contrast between slices, and hence not write any slice timing information to the image header. Perhaps I need to be more conservative by reporting that this capability applies for Siemens data only…
I’d also encourage Philips and GE users to put pressure on their vendors to push software updates to enable this. I’m surprised the fMRI community haven’t already done that, given the availability of software for slice timing correction.
… I didn’t know that I had to call
-cudaexplicitly in thedwipreproccommand (I thought it will find it automatically!)
Yes, currently it explicitly requires setting the -cuda option. At the time that script was first implemented I expected very few users to have gone to the effort of getting CUDA working, but many may have downloaded both binaries, and hence attempting to execute the CUDA version would lead to failures.
In the 3.0_RC3 update, detection of the CUDA eddy binary is more robust (searches for the highest CUDA version available, so you don’t have to explicitly construct an “eddy_cuda” softlink). I also now have mechanisms for catching failed command executions and allowing the script to continue. So I wouldn’t be against modifying dwipreproc to use the CUDA version by default if it can be found, if that would lead to less user issues / confusion. Thoughts?
This … still returns
UnicodeDecodeError: ‘utf8’ codec can’t decode byte 0x9e in position 1: invalid start byte
I’m trying to duplicate your use case as a way for me to simultaneously investigate the UTF-8 encoding issue. The only insight I’ve had so far has been in certain circumstances where the CUDA version of eddy generates a segmentation fault: the command outputs text relating to the memory areas implicated, but somewhere in that text is raw binary data that results in invalid codes if interpreted as UTF-8. Unfortunately this is not reproducible, which is always the first step in diagnosing and fixing faults.
… I’m thinking my best option may be to wait it out. Although, this may not solve the unicode issue…
Given this has cropped up a number of times now, the MRtrix3 update won’t be pushed out without some solution to the Unicode issue. Worst case scenario, terminal text output by some commands may not appear precisely as intended; but that’s a helluva lot better than users getting unhandled Python exception messages. Just trying to generate good reproducible use cases so that I can figure out the best solution. And make sure the solution works on all 3 OS’s, both versions of Python on each, and with and without the -debug option for each 
Hi Derek
I explain what happens to a couple of my subjects. It may or may not solve your issue but could be helpful for others with the same error.
In the dwipreproc temporary directory, there is a log file. In one of the lines in the middle, there is this command (in my case):
dwiextract dwi.mif -pe 0,1,0,0.0173 - | mrconvert - dwi_pe_1.nii -json_export dwi_pe_1.json
which creates a .json file. If I open this file (for some of my subjects), I see that there is “È” (E with a grave accent) or something like that in the subject’s name (coming originally from DICOMs/scanner) which in later steps causes ‘utf8’ problem when the .json file is used by:
mrconvert dwi_pe_1_topup.nii.gz dwi_pe_1_topup.mif -json_import dwi_pe_1.json
Traceback (most recent call last):
File “/home/…/mrtrix3/bin/dwipreproc”, line 488, in
run.command('mrconvert ’ + temp_path + ’ ’ + output_path + ’ -json_import ’ + json_path)
File “/home/…/mrtrix3/lib/mrtrix3/run.py”, line 183, in command
stderr_text = tempfiles[index][1].read().decode(‘utf-8’)
File “/usr/lib/python2.7/encodings/utf_8.py”, line 16, in decode
return codecs.utf_8_decode(input, errors, True)
UnicodeDecodeError: ‘utf8’ codec can’t decode byte 0xc9 in position 228: invalid continuation byte
dwipreproc:
So what I do is to replace this letter with normal ‘E’ (without accent grave) in the .json file, save the file and continue the dwipreproc as where it crashed.
Cheers,
Hamed
Check this out please:
http://dbic.dartmouth.edu/wiki/index.php/Slice_Acquisition_Order
Using above and knowing the ‘acquisition’ type from the DICOMs’ header (luckily they have included this 
), I guess it would be possible to address this issue.?
Cheers,
Hamed
I explain what happens to a couple of my subjects. It may or may not solve your issue but could be helpful for others with the same error.
While the reported error is the same, the underlying issue that is leading to the error differs quite significantly between your two cases (more info on GitHub).
@DAndrews: The way I was able to “reproduce” your issue was by providing an invalid slspec file, which leads to a CUDA segmentation fault; that segmentation fault then (sometimes) contains non-text data that causes the script library to have a hiccup. So in the absence of you actually seeing that error, I would suggest checking over your slpec file extremely closely; eddy does not appear to currently perform a sanity check on it, so if e.g. a slice is missing from the list, that will lead to your currently-rather-useless error feedback.
Check this out please:
I’ll have a look into that. I’m aware of the slice acquisition order fields in NIfTI, but I’d need to figure out how dcmstack is extracting that information from DICOM.
Ok, I went back and looked at my slspec.txt file. I had made a poor assumption that our slice acquisition started with slice “1”, when it actually starts with slice “0”.
After adjusting the slspec file, and taking the steps to copy the slspec file into the temp folder and use -continue dwipreproc and eddy_cuda run with no errors.
My test results look great! Thank you again Rob for all your help!
Best, Derek
Hi Rob,
Unfortunately, my celebration may have been a bit premature. While eddy_cuda appears to run and move on to the following mrconvert steps, then I get a general eddy_cuda failed error…
Command: eddy_cuda --imain=dwi.nii --mask=mask.nii --acqp=eddy_config.txt --index=eddy_indices.txt --bvecs=bvecs --bvals=bvals --repol --mporder=18 --s2v_niter=10 --s2v_interp=trilinear --s2v_lambda=5 --slspec=json_slspec.txt --out=dwi_post_eddy
Command: mrconvert dwi_post_eddy.nii.gz result.mif -stride -1,2,3,4 -fslgrad dwi_post_eddy.eddy_rotated_bvecs bvals
Command: mrconvert result.mif /data/proj/app/pipeline/DWI_TH/100604-200_3/100604-200_3_posteddy.mif.gz
dwipreproc: Changing back to original directory (/data/proj/app/pipeline/DWI_TH/100604-200_3)
dwipreproc: Script failed while executing the command: eddy_cuda --imain=dwi.nii --mask=mask.nii --acqp=eddy_config.txt --index=eddy_indices.txt --bvecs=bvecs --bvals=bvals --repol --mporder=18 --s2v_niter=10 --s2v_interp=trilinear --s2v_lambda=5 --slspec=json_slspec.txt --out=dwi_post_eddy
dwipreproc: For debugging, inspect contents of temporary directory: /data/proj/app/pipeline/DWI_TH/100604-200_3/dwipreproc-tmp-7ZN2XE
the output file is produced and looks good.
The error.txt file in the tmp directory gives the same old ‘msg=MultiBandGroups: Unable to open file error’ which I’m assuming is a hold over from the first run of dwipreproc that creates the tmp directory, which I then use -continue to flag. I’m also thinking this error.txt could be the reason for this error warning, when in fact all is well? I am using the -nocleanup flag…
The
error.txtfile in the tmp directory gives the same oldmsg=MultiBandGroups: Unable to open file errorwhich I’m assuming is a hold over from the first run ofdwipreprocthat creates the tmp directory, which I then use-continueto flag. I’m also thinking thiserror.txtcould be the reason for this error warning, when in fact all is well? I am using the-nocleanupflag…
Correct; all is indeed well. I only recently discovered this issue myself. It’s already fixed for the coming update.
Correct; all is indeed well.
Great news! I figured that was the issue. Best wishes with the rollout of the new update. Your guidance is very very appreciated!
Hi,
joining the ranks of problems occuring in dwipreproc associated with eddy_cuda. Since I’ve updated the latest MRtrix release candidate AND my macOS to High Sierra 10.13.4 (in one go) a couple of days ago, an error has kept occuring when running dwipreproc (the exact command was dwipreproc dwi_denoised.mif dwi_den_preproc.mif -rpe_pair -se_epi b0s.mif -pe_dir AP -eddy_options " --data_is_shelled --slm=linear ; and was run on unanonymized data), telling me that eddy_cuda_6.5 could not be carried out since I did not have the necessary permissions. I then reinstalled my FSL. Now I get the following message running the same command:
[WARNING] Command failed: eddy_cuda --imain=eddy_in.nii --mask=eddy_mask.nii --acqp=eddy_config.txt --index=eddy_indices.txt --bvecs=bvecs --bvals=bvals --topup=field --data_is_shelled --slm=linear --out=dwi_post_eddy
dwipreproc: [WARNING] CUDA version of eddy appears to have failed; trying OpenMP version
I also went back to a dataset I had preprocessed before the updates (without any warnings), using the exact same command as above. Now I’m getting the same warning there. Does anyone have any idea what the issue might be? Do you think that’s a problem of the MRtrix update or the OSX?
Best,
Marlene.
This sounds like an incompatibility problem between eddy and your OS and cuda similar to here. Does eddy_cuda work on any data? You might need to update cuda.
I had the same eddy_cuda error running dwipreproc on Amazon web services a couple days ago. I got around it by reinstalling cuda 7.5 toolkit and older nvidia-367 drivers which supported the GPU on the aws instance. Worked “almost” like a charm after that…
I had the Unicode error on 3 of 10 subjects during dwipreproc. It was a problem when mrtrix was reading the header. I had an older version of mrtrix still running on host machine (without cuda). So I ran dwipreproc and terminated it after it started the non-cuda Eddy. I just rsnyc’d the temp directory that dwipreproc created to AWS and manually executed the remaining dwipreproc commands (eddy_cuda, some mrconvert commands, and copying data from tmp directory back to original data folder). Annoying, but worked. I’ll have to see if the -continue switch works for me. That seems like an easier work around than what I did.
@maxpietsch I updated cuda, but the problem remains. Unfortunately, eddy_cuda doesn’t seem to work on any of my data any more.
@sniogi Which operating system are you running? The cuda 7.5 toolkit is not available for OSX 20.13.4, which I am now using. Neither are any of the nvidia-367 drivers I found. Do you have other suggestions?
To clarify: In 3.0_RC3, rather than explicitly requiring the -cuda command-line option in order to use the CUDA version of eddy, dwipreproc instead looks for the CUDA version and will automatically run that version. If however the CUDA version fails, then dwipreproc will issue a warning, and automatically run the OpenMP version (if it is installed). Therefore, if you have placed the CUDA version of eddy somewhere in your PATH but never attempted to run it, then with the 3.0_RC3 update you will now get a warning that that version is failing. If however the OpenMP version runs without issue, then the outcome from dwipreproc will not be detrimentally affected. If you are put off by the warning, the CUDA eddy binary should simply be deleted, since it’s unable to run on your system regardless.
CUDA is very sensitive to version mismatches; not only between the CUDA version installed on the system and that used to compile the binary (which is why there are multiple versions of the CUDA version of eddy provided), but also the kernel and video driver versions.
If you believe that the CUDA version should be working on your system, then you will need to run it manually and check the error message that it produces.
I had the Unicode error on 3 of 10 subjects during
dwipreproc.
I was hoping that Unicode conversion errors would have been removed with the 3.0_RC3 update. If you are indeed running this updated version and are still getting this error, please provide further details either here or on GitHub.
Thanks @rsmith - good to know OpenMP vs. Cuda doesn’t make a big difference. Dwipreproc finished fine on OpenMP 
@martahedl The AWS instance was running ubuntu 16.04. I have a Mac Pro workstation, but it doesn’t have an NVIDIA graphics card so no cuda locally for me.
@rsmith I just installed it on 5/17 so I assume it’s the latest version. This is the version output from mrconvert -version:
ubuntu:~$ mrconvert -version
== mrconvert 3.0_RC3-3-g17640ca9 ==
64 bit release version, built May 17 2018, using Eigen 3.2.92
Author(s): J-Donald Tournier (jdtournier@gmail.com) and Robert E. Smith (robert.smith@florey.edu.au)
Copyright © 2008-2018 the MRtrix3 contributors.
This Source Code Form is subject to the terms of the Mozilla Public
License, v. 2.0. If a copy of the MPL was not distributed with this
file, you can obtain one at http://mozilla.org/MPL/2.0/
MRtrix3 is distributed in the hope that it will be useful,
but WITHOUT ANY WARRANTY; without even the implied warranty
of MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.
For more details, see http://www.mrtrix.org/
This is the error I received:
ubuntu:~$ dwipreproc data2.mif test.mif -rpe_none -pe_dir AP -export_grad_fsl bvec.txt bval.txt -info
dwipreproc:
dwipreproc: Note that this script makes use of commands / algorithms that have relevant articles for citation; INCLUDING FROM EXTERNAL SOFTWARE PACKAGES. Please consult the help page (-help option) for more information.
dwipreproc:
dwipreproc: Loading header for image file ‘/home/ubuntu/s10/data2.mif’
Traceback (most recent call last):
File “/home/ubuntu/mrtrix3/bin/dwipreproc”, line 74, in
image.check3DNonunity(path.fromUser(app.args.input, False))
File “/home/ubuntu/mrtrix3/lib/mrtrix3/image.py”, line 101, in check3DNonunity
image_in = Header(image_in)
File “/home/ubuntu/mrtrix3/lib/mrtrix3/image.py”, line 20, in init
data = json.load(f)
File “/usr/lib/python2.7/json/init.py”, line 291, in load
**kw)
File “/usr/lib/python2.7/json/init.py”, line 339, in loads
return _default_decoder.decode(s)
File “/usr/lib/python2.7/json/decoder.py”, line 364, in decode
obj, end = self.raw_decode(s, idx=_w(s, 0).end())
File “/usr/lib/python2.7/json/decoder.py”, line 380, in raw_decode
obj, end = self.scan_once(s, idx)
UnicodeDecodeError: ‘utf8’ codec can’t decode byte 0xa0 in position 51: invalid start byte
Identical command on same data worked fine on an older release of mrtrix I had on my local machine. Let me know if you want me to post to GITHUB…
good to know OpenMP vs. Cuda doesn’t make a big difference.
Well, apart from the execution time, the other big difference is that slice-to-volume correction isn’t implemented in the OpenMP version of eddy. But given that you upgraded your OS at the same time, I would think that your inability to run eddy_cuda would be far more likely a result of that upgrade than the changes to MRtrix3.
This is the error I received:
Oooh OK. It’s the same Unicode error message as reported previously, but originating from a very different location. While I can see why these data may not have been a problem with a previous version of MRtrix3, it’s nevertheless not actually MRtrix3’s fault; at least not entirely. The error is arising deep inside of Python library code. But I might be able to provide an upstream solution…
What would be useful for me here, would be if you could run:
mrconvert data2.mif data2.mih
mrinfo data2.mif -json_all data2.json
, and upload files data2.mih and data2.json to the new GitHub issue I’ve made for this report. There’s no need to provide file data2.dat generated by the mrconvert call: it’s only the header information that I’m interested in. It’s important to explicitly upload the files rather than copy&pasting the text, as the latter process could obscure the issue.
You could also try running dwipreproc under Python3 and see whether it better deals with the data you have.
Dear all,
From where I can download OpenMP for macOS?
in the current version (3.0_RC3) i get an error:
dwipreproc: [WARNING] CUDA version of eddy appears to have failed; trying OpenMP version
You should not need to download OpenMP; it’s a library that the binary is compiled against. Assuming that you are only seeing the warning you have copy&pasted here and not a subsequent error, then the OpenMP version is working; it’s the CUDA version that is present on your system, but not successfully executing (probably because you downloaded the CUDA version of eddy but have not adequately set up CUDA itself).
The direct eddy downloads are here. However in checking the Mac downloads, it appears that the standard version of eddy is in fact called eddy_cpu there, rather than eddy_openmp. I had not realised this earlier, and so dwipreproc will not automatically detect this file. If this is the file that you have downloaded onto your system, and dwipreproc is failing to complete, creating a softlink called eddy that points to eddy_cpu should work. Alternatively, installing FSL version 6 using their installation script now includes eddy as part of its installation.
(Anyone who has used this script to install on Mac: please let me know if the names of the eddy executables are different there also)