I wanted to perform eddy using MRtrix’s dwifslpreproc command line and I gave it a go using a command similar to the first example given in the documentation. I knew it was going to take a couple hours so I just left it running while I did other stuff. However after 4 hours the console was still running (see pic) and I thought there was something wrong, so I killed it and started it again, and this time left it overnight. When I woke up this morning the issue was the same: the console was still running, even after 9 hours of execution.
I note these are ferret data. Can you provide information about the data, notably the dimensions of the data set, and also what hardware you’re running on? It might be that it’s simply a very big dataset. And if it’s running on the CPU, it will invariably take a lot longer than the GPU version. Furthermore, there’s a chance you might be running the single-threaded version of eddy, which will be slower again than the (multi-threaded) openMP version (I’d expect to see the command name as eddy-openmp normally – but I might be wrong here). I’m not sure whether 9h is reasonable given your data and hardware, but hopefully it’s nothing more sinister than that…
Indeed, I’m working with ferrets, but also sheep and mice, and some of my files are datasets with 2 b0’s and then 126 volumes at b=4000 in different directions. There are some datasets with fewer volumes, though, but many of them are like this one.
How can I instruct fsldwipreproc to run eddy in the GPU version or at least the multi-threaded one?
My hardware shouldn’t be a problem; I’m using a MacBook Pro with the following specs:
The OpenMP command appears at the end of the original screenshot, so this is probably the version that is running since it says it wasn’t successful when attempting CUDA.
There is clearly something going on and I don’t know what it is. Thoughts?
In another context I’d suggest running an MRtrix3 script with the -debug flag, as this takes the text outputs of the underlying commands being invoked and raises them to the terminal. However I’m pretty sure that eddy writes its progress information to stdout rather than stderr, which prevents this from working. If you were to instead go into the scratch directory and manually run that eddy command, perhaps also with the --verbose flag, there’s a chance it might tell you something.
Furthermore, there’s a chance you might be running the single-threaded version of eddy, which will be slower again than the (multi-threaded) openMP version (I’d expect to see the command name as eddy-openmp normally – but I might be wrong here).
I would note that in some instances myself I’ve observed eddy_openmp running single-threaded. It’s entirely possible it’s something specific to one of my own systems and not an issue for anyone else, but it’d be worth running top for a period of time to see if it’s succeeding in multi-threading (I think it will always run in a single thread at certain points during its execution, but it should change back to multi-threaded at some point).
How can I instruct fsldwipreproc to run eddy in the GPU version or at least the multi-threaded one?
dwifslpreproc has already attempted to run the GPU version of eddy. It was only after its failure that the CPU version was instead executed. There isn’t any way to “instruct” it to use CUDA: dwifslpreproc will attempt this version first if it’s present on your system. Since eddy_cuda is compiled for different CUDA versions, currently dwifslpreproc will select the version with the most recent CUDA version; unless your system also has a command called “eddy_cuda” with no CUDA version in the name (which e.g. might be a softlink to the version that works on your system), in which case that will instead be invoked.
As far as why it failed, you will find some information in the scratch directory in file “eddy_cuda_failure_output.txt”, which contains whatever eddy_cuda printed before it terminated.
After posting this question, I went on and tried eddy on its own with the --very_verbose flag, and discovered that eddy is indeed working in the background, so at least I can see that there is not an issue of the hardware getting “frozen” or anything. However, the process took 36 hours to be completed for 1 dataset. So I definitely need to look on how to fix this, as I have 45 datasets to analyse.
I followed your instructions and inspected the eddy_cuda_failure_output file, which stated the same as running dwifslpreproc with the -debug flag:
But I don’t know why this is the case. If there is no way as you mention of instructing eddy to use more threads, my guess is that my best choice is to fix the eddy_cuda execution problem somehow. Should I just “reinstall” FSL (i.e. re-download and execute the python script)?
Unfortunately, given you have an Intel GPU, you won’t be able to run the CUDA version – it only runs on Nvidia hardware.
Otherwise, I’m not sure why the openmp version isn’t multi-threading, but a quick search suggests that’s usually due to the software having been compiled without the right compiler settings, or it relying on libraries that themselves weren’t compiled to support openmp. Did you compile from source, or did you install the precompiled version? Might be worth looking through the FSL forum to see whether others have the same problem.
Thanks for your reply, J-Donald =) I also reached the same conclusion, and then I found that the issue is basically the new MacOS releases, since there isn’t an OpenMP version of eddy for these latest releases. Basically I’m screwed with my current setup. I’ll keep my eyes open for newer FSL releases to see how they go about this problem. In the meantime I’ll have to use other HPC approaches. Thanks for your time!
