Problems when trying to deploy MRtrix3 on an HPC cluster

1

Dear MRtrix3 developers,

I’m deploying MRtrix3 on an HPC cluster. Unfortunately, after trying three methods, I still failed to deploy it.
I would like to show the errors and problems I met as follow.
And I hope to get your help and advice.

1. Static build

I followed the steps on this site (Standalone installation on Linux).
At first, I tried to build MRtrix3 on my Macbook. When I was running this command ./configure -static -nogui , I met this error.

Looking for compiler [g++]: Apple clang version 14.0.0 (clang-1400.0.29.102)
Checking for C++11 compliance: test failed (see configure.log for details)

ERROR: no suitable compiler found!

 
And in configure.log, I got a detailed report.

EXEC <<
CMD: g++ /var/folders/m_/vrzdj7b14ks3mhq046yy2kxm0000gn/T/tmpjhobbl88.o -static -Wl,--whole-archive -lpthread -Wl,--no-whole-archive -mmacosx-version-min=12.6 -o a.out
EXIT: 1
STDERR:
ld: library not found for -lcrt0.o
clang: error: linker command failed with exit code 1 (use -v to see invocation)
>>

error deleting temporary file "a.out": No such file or directory
REPORT: test failed (see configure.log for details)

ERROR: no suitable compiler found!

 
After googling, I saw a similar problem (macos - ld: library not found for -lcrt0.o on OSX 10.6 with gcc/clang -static flag - Stack Overflow). However, I still couldn’t solve my problem after reading this page… :upside_down_face:

So I moved to a Ubuntu 22.04 OS and repeated this method. However, when I was running ./build, another error arose.

(191/397) [LB] bin/tckstats
(192/397) [LB] bin/mrthreshold

ERROR: (191/397) [LB] bin/tckstats

g++ tmp/core/formats/nifti1_gz.o tmp/core/formats/nifti2_gz.o tmp/core/image_io/ram.o tmp/core/math/erfinv.o tmp/core/mrtrix.o tmp/core/algo/histogram.o tmp/core/image_io/png.o tmp/core/file/dicom/element.o tmp/src/dwi/tractography/weights.o tmp/core/exception.o tmp/core/image_io/sparse.o tmp/core/header.o tmp/core/phase_encoding.o tmp/core/formats/png.o tmp/core/image_io/fetch_store.o tmp/core/file/png.o tmp/core/file/name_parser.o tmp/core/math/bessel.o tmp/core/dwi/gradient.o tmp/core/math/betainc.o tmp/core/file/key_value.o tmp/core/file/nifti_utils.o tmp/core/file/dicom/patient.o tmp/core/image_io/base.o tmp/core/formats/par.o tmp/core/file/dicom/select_cmdline.o tmp/src/dwi/tractography/roi.o tmp/core/file/mgh.o tmp/core/adapter/reslice.o tmp/core/dwi/shells.o tmp/core/formats/mgh.o tmp/core/file/mmap.o tmp/core/file/dicom/study.o tmp/core/file/config.o tmp/core/math/stats/import.o tmp/core/image_io/gz.o tmp/core/misc/bitset.o tmp/core/stride.o tmp/core/formats/xds.o tmp/core/file/tiff.o tmp/core/formats/mri.o tmp/core/math/SH.o tmp/core/math/stats/glm.o tmp/core/formats/dicom.o tmp/core/math/stats/fwe.o tmp/src/dwi/tractography/file_base.o tmp/core/file/dicom/image.o tmp/core/file/dicom/quick_scan.o tmp/core/file/dicom/dict.o tmp/core/formats/pipe.o tmp/core/axes.o tmp/core/formats/list.o tmp/core/file/dicom/series.o tmp/core/image_io/scratch.o tmp/core/image_io/pipe.o tmp/core/progressbar.o tmp/core/file/dicom/tree.o tmp/core/formats/nifti1.o tmp/core/file/dicom/mapper.o tmp/core/datatype.o tmp/core/image_io/mosaic.o tmp/core/file/json_utils.o tmp/cmd/tckstats.o tmp/tmp/tmpx94guhdu

This time, I had no idea what’s the problem, so I moved to next method.

2.Standalone packager

I followed the steps on this site (Standalone installation on Linux). Also, I tried this method on both Mac OS and Ubuntu OS.
Here is the problem on Mac OS when running ./package_mrtrix -standalone.

- collating MRtrix3 executables and library...
./package_mrtrix: line 62: cd: _package/mrtrix3/bin: No such file or directory
mv: rename _package to ../exe: Not a directory
mv: rename bin to ../exe: Not a directory
mv: rename build.default.active to ../exe: Not a directory
mv: rename cmd to ../exe: Not a directory
mv: rename core to ../exe: Not a directory
mv: rename docs to ../exe: Not a directory
mv: rename icons to ../exe: Not a directory
mv: rename lib to ../exe: Not a directory
mv: rename matlab to ../exe: Not a directory
mv: rename share to ../exe: Not a directory
mv: rename src to ../exe: Not a directory
mv: rename testing to ../exe: Not a directory
mv: rename tmp to ../exe: Not a directory
- collating dependent system libraries...
./package_mrtrix: line 72: ldd: command not found
./package_mrtrix: line 72: ldd: command not found
./package_mrtrix: line 72: ldd: command not found
./package_mrtrix: line 72: ldd: command not found
./package_mrtrix: line 72: ldd: command not found
./package_mrtrix: line 72: ldd: command not found
./package_mrtrix: line 72: ldd: command not found
./package_mrtrix: line 72: ldd: command not found
./package_mrtrix: line 72: ldd: command not found
./package_mrtrix: line 72: ldd: command not found
./package_mrtrix: line 72: ldd: command not found
./package_mrtrix: line 72: ldd: command not found
./package_mrtrix: line 72: ldd: command not found
./package_mrtrix: line 72: ldd: command not found
./package_mrtrix: line 72: ldd: command not found
./package_mrtrix: line 72: ldd: command not found
./package_mrtrix: line 72: ldd: command not found
./package_mrtrix: line 72: ldd: command not found
./package_mrtrix: line 72: ldd: command not found
./package_mrtrix: line 72: ldd: command not found
./package_mrtrix: line 72: ldd: command not found
./package_mrtrix: line 72: ldd: command not found
./package_mrtrix: line 72: ldd: command not found
./package_mrtrix: line 72: ldd: command not found
./package_mrtrix: line 72: ldd: command not found
./package_mrtrix: line 72: ldd: command not found
./package_mrtrix: line 72: ldd: command not found
./package_mrtrix: line 72: ldd: command not found
./package_mrtrix: line 72: ldd: command not found
./package_mrtrix: line 72: ldd: command not found
./package_mrtrix: line 72: ldd: command not found
./package_mrtrix: line 72: ldd: command not found
./package_mrtrix: line 72: ldd: command not found
./package_mrtrix: line 72: ldd: command not found
./package_mrtrix: line 72: ldd: command not found
./package_mrtrix: line 72: ldd: command not found
./package_mrtrix: line 72: ldd: command not found
./package_mrtrix: line 72: ldd: command not found
./package_mrtrix: line 72: ldd: command not found
./package_mrtrix: line 72: ldd: command not found
./package_mrtrix: line 72: ldd: command not found
./package_mrtrix: line 72: ldd: command not found
./package_mrtrix: line 72: ldd: command not found
./package_mrtrix: line 72: ldd: command not found
./package_mrtrix: line 72: ldd: command not found
./package_mrtrix: line 72: ldd: command not found
./package_mrtrix: line 72: ldd: command not found
./package_mrtrix: line 72: ldd: command not found
./package_mrtrix: line 72: ldd: command not found
./package_mrtrix: line 72: ldd: command not found
./package_mrtrix: line 72: ldd: command not found
./package_mrtrix: line 72: ldd: command not found
./package_mrtrix: line 72: ldd: command not found
./package_mrtrix: line 72: ldd: command not found
./package_mrtrix: line 72: ldd: command not found
./package_mrtrix: line 72: ldd: command not found
./package_mrtrix: line 72: ldd: command not found
./package_mrtrix: line 72: ldd: command not found
./package_mrtrix: line 72: ldd: command not found
./package_mrtrix: line 72: ldd: command not found
./package_mrtrix: line 72: ldd: command not found
./package_mrtrix: line 72: ldd: command not found
./package_mrtrix: line 72: ldd: command not found
./package_mrtrix: line 72: ldd: command not found
./package_mrtrix: line 72: ldd: command not found
./package_mrtrix: line 72: ldd: command not found
./package_mrtrix: line 72: ldd: command not found
./package_mrtrix: line 72: ldd: command not found
./package_mrtrix: line 72: ldd: command not found
./package_mrtrix: line 72: ldd: command not found
./package_mrtrix: line 72: ldd: command not found
./package_mrtrix: line 72: ldd: command not found
./package_mrtrix: line 72: ldd: command not found
./package_mrtrix: line 72: ldd: command not found
./package_mrtrix: line 72: ldd: command not found
./package_mrtrix: line 72: ldd: command not found
./package_mrtrix: line 72: ldd: command not found
./package_mrtrix: line 72: ldd: command not found
./package_mrtrix: line 72: ldd: command not found
./package_mrtrix: line 72: ldd: command not found
./package_mrtrix: line 72: ldd: command not found
./package_mrtrix: line 72: ldd: command not found
./package_mrtrix: line 72: ldd: command not found
./package_mrtrix: line 72: ldd: command not found
./package_mrtrix: line 72: ldd: command not found
./package_mrtrix: line 72: ldd: command not found
./package_mrtrix: line 72: ldd: command not found
./package_mrtrix: line 72: ldd: command not found
./package_mrtrix: line 72: ldd: command not found
./package_mrtrix: line 72: ldd: command not found
./package_mrtrix: line 72: ldd: command not found
./package_mrtrix: line 72: ldd: command not found
./package_mrtrix: line 72: ldd: command not found
./package_mrtrix: line 72: ldd: command not found
./package_mrtrix: line 72: ldd: command not found
./package_mrtrix: line 72: ldd: command not found
./package_mrtrix: line 72: ldd: command not found
./package_mrtrix: line 72: ldd: command not found
./package_mrtrix: line 72: ldd: command not found
./package_mrtrix: line 72: ldd: command not found
./package_mrtrix: line 72: ldd: command not found
./package_mrtrix: line 72: ldd: command not found
./package_mrtrix: line 72: ldd: command not found
./package_mrtrix: line 72: ldd: command not found
./package_mrtrix: line 72: ldd: command not found
./package_mrtrix: line 72: ldd: command not found
./package_mrtrix: line 72: ldd: command not found
./package_mrtrix: line 72: ldd: command not found
./package_mrtrix: line 72: ldd: command not found
./package_mrtrix: line 72: ldd: command not found
./package_mrtrix: line 72: ldd: command not found
./package_mrtrix: line 72: ldd: command not found
./package_mrtrix: line 72: ldd: command not found
./package_mrtrix: line 72: ldd: command not found
./package_mrtrix: line 72: ldd: command not found
./package_mrtrix: line 72: ldd: command not found
./package_mrtrix: line 72: ldd: command not found
./package_mrtrix: line 72: ldd: command not found
./package_mrtrix: line 72: ldd: command not found
./package_mrtrix: line 72: ldd: command not found
./package_mrtrix: line 72: ldd: command not found
./package_mrtrix: line 72: ldd: command not found
./package_mrtrix: line 72: ldd: command not found
./package_mrtrix: line 72: ldd: command not found
./package_mrtrix: line 72: ldd: command not found
./package_mrtrix: line 72: ldd: command not found
./package_mrtrix: line 72: ldd: command not found
./package_mrtrix: line 72: ldd: command not found
./package_mrtrix: line 72: ldd: command not found
    libraries identified:

It didn’t surprise me for the command ldd cannot run on Mac OS. But I couldn’t figure out this error ./package_mrtrix: line 62: cd: _package/mrtrix3/bin: No such file or directory. So I moved to Ubuntu.

Here is the problem on Ubuntu OS when running ./package_mrtrix -standalone.

Press enter to proceed (or Ctrl-C to abort)...
- collating MRtrix3 executables and library...
- collating dependent system libraries...
	not a dynamic executable
	not a dynamic executable
	not a dynamic executable
	not a dynamic executable
	not a dynamic executable
	not a dynamic executable
	not a dynamic executable
	not a dynamic executable
	not a dynamic executable
	not a dynamic executable
	not a dynamic executable
	not a dynamic executable
	not a dynamic executable
	not a dynamic executable
	not a dynamic executable
	not a dynamic executable
	not a dynamic executable
	not a dynamic executable
    libraries identified:
        /lib64/ld-linux-x86-64.so.2
        /lib/x86_64-linux-gnu/libc.so.6
        /lib/x86_64-linux-gnu/libdeflate.so.0
        /lib/x86_64-linux-gnu/libfftw3.so.3
        /lib/x86_64-linux-gnu/libgcc_s.so.1
        /lib/x86_64-linux-gnu/libjbig.so.0
        /lib/x86_64-linux-gnu/libjpeg.so.8
        /lib/x86_64-linux-gnu/liblzma.so.5
        /lib/x86_64-linux-gnu/libm.so.6
        /lib/x86_64-linux-gnu/libpng16.so.16
        /lib/x86_64-linux-gnu/libstdc++.so.6
        /lib/x86_64-linux-gnu/libtiff.so.5
        /lib/x86_64-linux-gnu/libwebp.so.7
        /lib/x86_64-linux-gnu/libz.so.1
        /lib/x86_64-linux-gnu/libzstd.so.1

At first, I thought it isn’t a big problem. So I copy the resulting _package/mrtrix3 folder to my HPC cluster and found I was wrong.

$ ./mrinfo
./mrinfo: error while loading shared libraries: libpng16.so.16: cannot open shared object file: No such file or directory

It seems that some libraries didn’t create during ./package_mrtrix -standalone. I have no idea how to deal with it either.

3.Install MRtrix3 and all its dependencies from source

I followed the instruction on this site (Installation of MRtrix3 and all its dependencies from source).
My HPC cluster already has g++ and python environment. So I started from getting Eigen3 from http://bitbucket.org/eigen/eigen/get/3.2.8.tar.gz, however, this resource seemed invalid.
Therefore, I turned to the official website of Eigen3 and got eigen3.4.0 from https://gitlab.com/libeigen/eigen/-/archive/3.4.0/eigen-3.4.0.tar.gz.
When running ./configure -nogui on my HPC cluster, The script couldn’t find eigen3 though I had already set the path for eigen3 by export EIGEN_CFLAGS="-isystem /path/to/eigen3/"

Thanks!

All the best,
Volcano

1 Like
2

Thanks to the administrator of my HPC cluster, I successfully built MRtrix3 on the HPC cluster by the third method. The problem that I reported is wrong. Actually, from where I stand, the real problem is the version between gcc(10.2.0) and eigen3(3.4.0) does not match according to configure.log. After the administrator of my HPC cluster provided a lower version of gcc(6.3.0), I successfully deployed MRtrix3 on my HPC cluster.

Still, I want to report the errors mentioned in the first and second methods. I think figuring these problems out will help a lot.

All the best,
Volcano

1 Like
3

OK, this is always really tricky, and I’m not surprised things have been failing.

That said, there is no way you can build packages on macOS and expect them to run on your HPC (which is presumably running some flavour of Linux) – they’re different architectures. Even if you had managed to produce static or standalone builds on your mac, they would never have run on your HPC.

For this reason, there is an implicit assumption in the instructions for the static and standalone builds that this is being done on a Linux system, hopefully as similar to the HPC environment as possible. The instructions would likely need to change to work on mac (but there’s little point in doing that, for the previous reason).

For your last point, I think you should have been able to run the build to completion simply by running ./configure again – though there is an issue with eigen 3.4.0 that has been fixed on master, but hasn’t made it to release yet (note to self: really need to push out a new release…). The simplest fix if that’s the issue is to use the more recent eigen release from the 3.3 branch.

In any case, glad you got it working. :+1:

4

Thanks for your reply!

Actually, I have tried these methods on both Mac OS and Linux OS(Ubuntu22.04). The errors that arose on Linux OS are as below.

And before I moved the package of MRtrix3 to the HPC cluster, these problems showed up when I was static building or packaging it.

In order to eliminate all distractions, I started building on a new computer with Linux OS following the installation steps from the doc. So I think maybe there are some bugs. Maybe because of the version?

And I think figuring out these problems is meaningful and maybe doing good for other users and future works so I hope developers can take a look at it if they have time. :smile:

Besides, I would like to thank you and all the developers of MRtrix3 for this powerful tool and an active community. :smiling_face_with_three_hearts:

5

Ok, sorry, got the wrong end of the stick there… I’ll have a look into it – though the chances are I’ll probably end up simply deleting these instructions… Static builds are not very well supported by modern compilers, they’re rarely used. The standalone build should work ok though, I’ll investigate when I have a minute.