Pathing different mrtrix functions

Hi all, I’m trying to figure out how to call different mrtrix functions located in a separate dev branch install but am not very familiar with this process.

When I call the default mrtrix (no dev branch) functions, they appear to be in the anaconda binaries dir on our filesystem, e.g., …/anaconda3/bin/tensor2metric

The system admin installed the dev branch in a separate location /software/mrtrix3/bin but when I add this to my local path, shell commands still seem to call the original functions in the anaconda dir. I’ve tried calling the dev branch functions specifying the full filepath as suggested here, but am getting the message error while loading shared libraries: libmrtrix.so (this file is in /anaconda3/lib)

Does anyone have advice on calling the alternate versions of normal vs. dev branch? This seems like a very basic problem I’m having but I don’t have much background here. Many thanks!

Hi @ewing,

OK, these issues can indeed get pretty messy – especially when anaconda is in the mix, since it makes all kinds of subtle behinds-the-scenes changes that can be pretty difficult to track down…

There are really two issues here, but they may be the same issue – it depends on how clever the system is. Let’s start with the second one, since that one really should work properly; that’s the one with ‘error while loading shared libraries’.

First thing to check, which libraries does tensor2metric require:

$ ldd /software/mrtrix3/bin/tensor2metric

On my system, this gives:

$ ldd ~/mrtrix3/bin/tensor2metric 
	linux-vdso.so.1 (0x00007f9ccfcc7000)
	libmrtrix.so => /home/donald/mrtrix3/bin/../lib/libmrtrix.so (0x00007f9ccf800000)
	libstdc++.so.6 => /usr/lib/libstdc++.so.6 (0x00007f9ccf400000)
	libm.so.6 => /usr/lib/libm.so.6 (0x00007f9ccfb6c000)
	libgcc_s.so.1 => /usr/lib/libgcc_s.so.1 (0x00007f9ccfb47000)
	libc.so.6 => /usr/lib/libc.so.6 (0x00007f9ccf21e000)
	/lib64/ld-linux-x86-64.so.2 => /usr/lib64/ld-linux-x86-64.so.2 (0x00007f9ccfcc9000)
	libz.so.1 => /usr/lib/libz.so.1 (0x00007f9ccfb2d000)
	libtiff.so.6 => /usr/lib/libtiff.so.6 (0x00007f9ccf772000)
	libpng16.so.16 => /usr/lib/libpng16.so.16 (0x00007f9ccf738000)
	libzstd.so.1 => /usr/lib/libzstd.so.1 (0x00007f9ccf14b000)
	liblzma.so.5 => /usr/lib/liblzma.so.5 (0x00007f9ccf705000)
	libjbig.so.2.1 => /usr/lib/libjbig.so.2.1 (0x00007f9ccfb1e000)
	libjpeg.so.8 => /usr/lib/libjpeg.so.8 (0x00007f9ccf09f000)

In your case, I expect it’ll report:

	libmrtrix.so => not found

somewhere in the mix…

Assuming that’s the case, we next need to figure out where the system is looking for libraries when loading the executable. There are two sources for this information: the system linker search path, and the executable’s own runpath (which we have baked into the executable itself). To query these, you can use:

  • For the system linker search path (this is for reference, it really shouldn’t be relevant here):

    ldconfig -v 2>/dev/null | grep ^/
    

    On my system, this gives:

    $ ldconfig -v 2>/dev/null | grep ^/
    /usr/lib/R/lib: (from /etc/ld.so.conf.d/R.conf:1)
    /usr/lib/libfakeroot: (from /etc/ld.so.conf.d/fakeroot.conf:1)
    /usr/lib/octave/8.3.0: (from /etc/ld.so.conf.d/octave.conf:1)
    /usr/lib: (from <builtin>:0)
    

    These are the folders where the system will look for all the different libraries. If you look in those folders, you should find a large number of .so files – these are libraries, aka shared objects – what would be called a DLL (Dynamically Linked Library) on Windows, or a .dylib (Dynamic Library) on macOS.

    You really shouldn’t find libmrtrix.so in those folders, unless someone has done something really strange…

  • for the executable’s runpath (this is the relevant one for MRtrix):

    readelf -d /software/mrtrix3/bin/tensor2metric | grep RUNPATH
    

    On my system, this gives:

    $ readelf -d ~/mrtrix3/bin/tensor2metric | grep RUNPATH
     0x000000000000001d (RUNPATH)            Library runpath: [$ORIGIN/../lib]
    

    which is as expected: we have compiled our code to expect the MRtrix library to be located in the lib/ folder next to the command’s own bin/ folder – i.e. in .../lib relative to the executable.

At this point, you can look in the folder specified in the runpath, which in your case can be done with:

ls -l /software/mrtrix3/lib

On my system, this gives:

$ ls -l ~/mrtrix3/lib/
total 3572
-rwxr-xr-x 1 donald donald 3653096 Oct  9 13:53 libmrtrix.so
drwxr-xr-x 7 donald donald    4096 Jan 19  2023 mrtrix3

And you can see that the libmrtrix.so file is indeed in the expected location (along with the mrtrix3 folder, which contains our Python libraries).

The final bit to check is that the permissions for the MRtrix library are set correctly, so it is marked as executable – otherwise the system won’t allow the code it contains to be executed. In the above you can see the 3 “x” in the permissions section of the output for libmrtrix.so, which indicates that it’s executable for everyone on the system (see e.g. here for a breakdown on how to interpret Unix permissions).

If all of the above checks out, then I really can’t understand why this doesn’t work… But my guess is your sysadmin may have omitted to copy over the lib/ folder when installing (see this document for details on deployment), or somehow reset the permissions on the library when copying it across. But you’ll need to look for yourself to find out…


For the other issue (that it defaults back to the anaconda install), one possibility might be that the system is smart enough to not even try to run the new version as it can already detect that it can’t find the libmrtrix.so library, and so skips to the next available executable it can find with the same name…

Otherwise, there is a faint chance you may not have set the PATH correctly – check with:

$ echo $PATH

Note that it needs to contain the /software/mrtrix3/bin folder – not just /software/mrtrix3. But I doubt that’s the issue – worth double-checking nonetheless…

OK, that should already give you plenty of information to try to figure this out. Let me know how you go!

All the best,
Donald.

Thanks so much for the detailed advice on this-- it’s been very helpful. I’ve gone through all of the steps and the issue seems to be related to the absence of a lib/ folder where the dev branch is installed-- i.e., /software/mrtrix3/ only contains a bin/ subdirectory and nothing else.

The libmrtrix.so file that the dev branch tensor2metric cannot find is only in the general /anaconda3/lib/ directory, which I guess contains many non-mrtrix files as well. When I try which tensor2metric the default is /software/anaconda3/bin/tensor2metric. So if I try and fullpath the command to the dev branch version /software/mrtrix3/bin/tensor2metric, it would need a lib/ copy in that /software/mrtrix3/ directory even if my path contains the directory with the .so file (/anaconda3/lib/)?

Given that the mrtrix-related lib files seem like the are merged with other files in /anaconda3/lib, is it a bad idea to just copy that whole dir to the dev branch build (are the lib files actually different between the normal install and the dev branch versions for that matter?) If so, perhaps the move is to rebuild either the base or dev branch version somewhere else and just copy the mrtrix-only lib directory to /software/mrtrix3/.

Hopefully this isn’t more confusing. I haven’t done either of the installs so I’m trying to piece it together from the deployment page and the wiki post on the dev branch. Thanks again for helping me troubleshoot.

OK, so that all makes sense.

Yes, but you can’t use the anaconda one. In fact, if you did manage to get the system to try using that libmrtrix.so, it would most likely fail with a ‘version mismatch’ error. The reason is that we frequently make changes to both the mrtrix library and the executables in the course of development. The executables can only be guaranteed to work if they use the library that was compiled from the same source tree (the same branch and version). The best way we’ve found to guarantee this is by using the runpath (as described above) to ensure the executables use the library that is co-located with the executables, but we additionally perform a version check at runtime to ensure the version of the executables matches the version of the library exactly. That will fail if you try to use a library from a different build (such as the anaconda-supplied one).

Yes! See above.

Something like that, yes. You can ask your sys admin if they still have the build folder available, and if so copy over the remaining files required (the whole lib/ and share/ folders). Alternatively, you can try to build this yourself by following our instructions – and in this case you can just clone and build within your own home folder, e.g. in ~/mrtrix3-dev. There is no need for it to be placed in /software/mrtrix3 particularly, unless you want that particular version of the dev branch to be available to other users as well.

In fact, given that dev is a moving target, I would assume most interested users will probably want to maintain their own build. Otherwise you may have to keep asking your IT admins to go through this again when/if you need to use a new feature on dev

All the best,
Donald

Great, that all makes sense-- best approach for the dev branch definitely seems to be an individual/user-specific build so I’m going to give that a shot.

Just to test, I did ask that the dev branch lib folder be copied to the correct location as you suggested and the function I was trying to use–MK output from tensor2metric-- now runs (the output MK output is actually a blank volume but that’s a different topic). These installation details are helpful-- the setup makes more sense now and I might see if we can organize the shared/user-specific installs differently here going forward.

Going to troubleshoot the DKI and see if I can figure out that aspect but I think the dev branch is working-- many thanks for the help!

1 Like