OK, there are multiple issues going on, so there’s quite a bit to untangle. Please bear with me, I’ll try to work out what’s going on first, then we can look into how to get things working properly. I’ll go into a fair bit of detail in the hope that this also helps others who come across similar issues, but hopefully you won’t need to follow everything to get the point.
Issue 1: Unable to locate MRtrix3 Python modules
Your original issue relates to our Python handling. This is very unlikely to be anything that can be fixed by configure & build. To understand why, I’ll need to explain how the code is organised.
MRtrix consists of a collection of C++ & Python commands. C++ commands need to be compiled from the source code that we write, into binary (machine) code that the computer can execute. We use C++ for anything that requires high performance. The configure & build scripts relate exclusively to the process of compiling the C++ executables – they have no impact whatsoever on the Python commands.
The Python commands are simple Python scripts, which don’t need to be compiled. They are executed directly by the Python interpreter, which does the job of translating the instructions in the script on the fly (or at least, using just-in-time compilation behind the scenes). In MRtrix, these commands need to locate the Python libraries (modules), which contain common code to perform the various operations that these scripts require. These Python libraries are located in the
lib/mrtrix3/ folder (you’re welcome to have a look).
The problem you’re having is that in your case, the mechanism that each Python command uses to locate the corresponding libraries is failing. We use a slightly unorthodox process to handle this aspect of things to ensure the correct version of the libraries/modules is used every time, using a Python module co-located with the commands in the
bin/ folder, called
mrtrix3.py. Its job is to look at where the command you’re trying to run is located (hopefully in your installation’s
bin/ folder), and see if it can find the modules with the same version of the code in the
lib/ folder, which it should find alongside the command’s
bin/ folder. The message you’re seeing is precisely the error that this part of the code throws up when it can’t locate the expected modules.
What to do about it
I would start by investigating your installation and making sure that everything is as expected. First, where is the installation that you’re actually using:
This will hopefully print out the folder where it found the
dwifslpreproc command. e.g. on my system, I get:
$ which dwifslpreproc
which tells me that my installation is located in
/home/donald/mrtrix3/, and the command is indeed in the
bin/ folder within that installation.
Next, change directory to your installation folder, and list the contents of the
lib/mrtrix3/ folder. E.g. for my example above:
$ cd /home/donald/mrtrix3/
$ ls lib/mrtrix3/
_5ttgen dwibiascorrect __init__.pyc __pycache__ _version.py
algorithm.py dwinormalise matrix.py run.py _version.pyc
algorithm.pyc fsl.py matrix.pyc run.pyc
app.py image.py path.py sh.py
app.pyc image.pyc path.pyc utils.py
dwi2response __init__.py phaseencoding.py utils.pyc
If you get a ‘file not found’ error, then your installation is not complete, or has somehow been corrupted. That folder must exist for the scripts to work.
If it does list the folder contents, but you don’t see at least the
__init__.py file, then that would also lead to this kind of error (this file is how Python figures out that the entire folder is to be treated as a single Python module).
If everything checks out, and you do see the same folder contents as I do (you can ignore the
.pyc files, by the way), then we’ll need to dig deeper. But assuming you do find this is the issue, then the next stage is to figure out what happened for you to end up with an incomplete installation. Maybe someone copied the MRtrix commands from the
bin/ folder to a different location to make them available to everyone else, not realising that the presence of the associated
share/) folders in the expected locations relative to these commands is important? If so, you can point them towards our documentation on deployment. But otherwise, you’ll have to work what happened from your side, and figure out whether this is something that can be fixed easily (e.g. you may just need to copy the
lib/ folder over from where the code was originally compiled), or whether you really need to rebuild everything…
Issue 2: Failure to configure & build
Hopefully, the above will sort out your original issue. But it may be that you still need to rebuild to code for some other reason (or maybe you just want to). In this case, read on.
First off, it looks like when you start a terminal session, you’re running within an anaconda environment. While that’s not a problem when simply running MRtrix commands, it is a problem when trying to build MRtrix, because anaconda overrides the system compiler and tries to get the build to use the packages it provides, rather than the packages your Linux distribution (in your case, Ubuntu) provides for you. The issues with that is that it’s near-impossible to compile MRtrix with the anaconda packages, particularly the graphical apps (
shview) – presumably because you have to use system packages to display anything (needs access to the specific drivers for your exact GPU hardware – not something anaconda can supply for everyone).
There’s an additional conflict with FSL, as it now provides a full-fledged installation of Qt5 (if someone can shed some light as to why, that would be most welcome…), and that also conflicts with your system packages.
I can see both of these issues happens when you try to configure: it’s trying to use the anaconda compiler:
Looking for compiler [g++]: g++ (conda-forge gcc 11.3.0-19) 11.3.0
and trying to compile against anaconda-supplied packages (note the location of these libraries is in your
This also happens for libPNG and libTIFF.
You can see the conflict with FSL when it’s trying to find the Qt5 libraries (required for the graphical apps):
So as you can see, lots of reasons for conflicts. These are not things that we can control on user’s systems, by the way. Anaconda and FSL have both decided to override critical aspects of the environment in a way that will introduce deep conflicts of this type. I’m not convinced I understand the rationale for any of this, personally, but it is what it is, and we now have to deal with the consequences.
What to do about it
To avoid these conflicts, it should be sufficient to run a terminal session with no trace of either anaconda or FSL. Since these are typically imported into your terminal session when your startup files are read (when your shell starts up), the simplest way to avoid this is to start a
bash session where we explicitly request that your startup files are not used. This is what this command does:
bash --noprofile --norc
Note that this has not impact on the Python module issue – that’s a different problem. This is purely to ensure our configure & build scripts don’t come into conflict with anaconda or FSL.
If that doesn’t work, an alternative is to modify your startup scripts (presumably
~/.profile, but potentially also
~/.bash_profile – and others again if you don’t use the
bash shell…), and comment out the lines responsible for setting up the anaconda and FSL environments. You should then be able to start a fresh terminal that is ‘unaware’ of these packages, and run the configure & build there. Once done, you can always undo the modifications to your startup scripts to recover your original environment.
Indeed, in one of your previous posts, it looks like the configure script managed to run to completion. As I’d noted in response to that post, this resulted in different versions of the packages being reported, notably Qt 5.5.0 rather than 5.15.6 as was the case previously. This is hopefully because the configure script found your system libraries (version 5.5.0), not the FSL-supplied ones. It also found your system compiler (
g++ (Ubuntu 5.5.0-12ubuntu1~16.04) 5.5.0 20171010), not the anaconda compiler (
g++ (conda-forge gcc 11.3.0-19) 11.3.0).
I note that for some reason, it still found anaconda in the PATH, as reported in this warning:
which is disappointing – I wasn’t expecting to see any hint of it. But it looks like the rest of the configure script nonetheless managed to ignore it, so hopefully that was all that was needed.
I suggest you try again with that procedure, and keep going if there is no overt error. This should look like:
$ bash --noprofile --norc
$ cd mrtrix3 # <-- if needed
If that still doesn’t work, feel free to report back with the full output of each command, and the full contents of the
./build.log (if you got that far).
Hope this helps!
All the best,