Permission denied when using mrconvert and dwi2response?

Hi,
While executing mrconvert (see below), I get a permission denied error ??? but the DWI.mif file gets created:

mrconvert before_MR/DWIs_upscaled.nii.gz DWI.mif -grad mr_btable.txt -datatype float32 -stride 0,0,0,1
mrconvert: [100%] uncompressing image "before_MR/DWIs_upscaled.nii.gz"
mrconvert: [ERROR] error opening file "DWI.mif": Permission denied

So, then I tried the command dwi2response (even with sudo) and I get the following error:

sudo dwi2response fa DWI.mif  response_func.txt
Password:
dwi2response: 
dwi2response: Note that this script makes use of commands / algorithms that have relevant articles for citation. Please consult the help page (-help option) for more information.
dwi2response: 
dwi2response: Generated temporary directory: /tmp/dwi2response-tmp-H9CJT0/
Command: mrconvert /<DIR_LOCATION>/DWI.mif /tmp/dwi2response-tmp-H9CJT0/dwi.mif -stride 0,0,0,1
dwi2response: 
dwi2response: [ERROR] Command failed: mrconvert  /<DIR_LOCATION>/DWI.mif /tmp/dwi2response-tmp-H9CJT0/dwi.mif -stride 0,0,0,1
dwi2response: Output of failed command:
mrconvert: [ERROR] error opening file "/tmp/dwi2response-tmp-H9CJT0/dwi.mif": Permission denied
dwi2response: Changing back to original directory (/ /<DIR_LOCATION>)
dwi2response: Script failed while executing the command: mrconvert  /<DIR_LOCATION>/DWI.mif /tmp/dwi2response-tmp-H9CJT0/dwi.mif -stride 0,0,0,1
dwi2response: For debugging, inspect contents of temporary directory: /tmp/dwi2response-tmp-H9CJT0/

Any suggestions on why this is happening?

My mr version:
The MRtrix viewer, version 0.3.15-528-g00f9837e
64 bit release version, built Apr 6 2017

… but the DWI.mif file gets created

Does the file have a size that’s expected given the image contents? It’s possible that the initial file creation step succeeds, but the resize step (which allocates storage for the raw image data over and above the size of the header) fails.

Are your working directory and /tmp/ part of the same filesystem? Is it possible that you have simply run out of storage space? (This can result in entirely non-informative errors sometimes.)

Never seen anything like this… But I agree with @rsmith:

There’s a few steps that take place one after the other, and one of them might be failing. My money was on the final memory-mapping stage (it’s possible that the particular file-system you’re using doesn’t support memory mapping, for instance). If you run with the -debug option, we should be able to figure out which step is failing. You should see this output for a standard mrconvert run:

$ mrconvert in.mif test.mif
...
mrconvert: [INFO] creating image "test.mif"...
mrconvert: [DEBUG] sanitising image information...
mrconvert: [DEBUG] creating empty file "test.mif"
mrconvert: [DEBUG] resizing file "test.mif" to 195085088
mrconvert: [DEBUG] sanitising image information...
mrconvert: [DEBUG] memory-mapping file "test.mif"...
mrconvert: [DEBUG] file "test.mif" mapped at 0x7f1ecedba000, size 195077376 (read-write)
mrconvert: [DEBUG] image "test.mif" loaded
mrconvert: [DEBUG] image "test.mif" initialised with strides = [ -1 -104 10816 584064 ], start = 10815, using indirect IO
...

You can see the creation, resize, and final memory-mapping operations being reported here. Might give us a better clue as to what is going on…

Also critically important: what platform is this on? Which OS / distribution / version, and most important, what filesystem is this file being written to? You can use the df -T . command for this last step, running within the target folder.

Thank you all for your prompt response. Now I’ve decided to reinstall everything from scratch as I previously had storage issues (and now libc++ issues, check below). I’ll keep you posted.

Regarding my previous problems

@rsmith:

  1. The file size seems correct, DWI.mig is 1.1G while DWI_orig.nii.gz is 1009MB.
  2. Yes, /tmp and my working directory are of the same filesystem. I even tried changing my /tmp directory to my working directory (I think I did that in the dwi2response script).
  3. Storage issues shouldn’t be a problem (I have over 600GB).

@jdtournier:
(sorry I should’ve included this before)
I ran this on a OSX 10.11.6 (el capitan). Now I just updated to Sierra (so OSX 10.12.4) to see if ths helps but I get a different error. The filesystem is: /dev/disk which is the only partition I have in my Macbook Pro

Now running:

$ mrconvert before_MR/DWIs_upscaled.nii.gz DWI.mif -grad mr_btable.txt -datatype float32 -stride 0,0,0,1 -debug
mrconvert: [DEBUG] No config file found at "/etc/mrtrix.conf"
mrconvert: [DEBUG] No config file found at ".mrtrix.conf"
mrconvert: [INFO] opening image "before_MR/DWIs_upscaled.nii.gz"...
mrconvert: [DEBUG] sanitising image information...
mrconvert: [DEBUG] searching for suitable gradient encoding...
mrconvert: [DEBUG] loading matrix file "mr_btable.txt"...
mrconvert: [DEBUG] found 35x4 matrix in file "mr_btable.txt"
mrconvert: [INFO] found 35x4 diffusion gradient table
mrconvert: [DEBUG] loading image "before_MR/DWIs_upscaled.nii.gz"...
mrconvert: [100%] uncompressing image "before_MR/DWIs_upscaled.nii.gz"
mrconvert: [DEBUG] image "before_MR/DWIs_upscaled.nii.gz" loaded
mrconvert: [DEBUG] image "before_MR/DWIs_upscaled.nii.gz" initialised with strides = [ 1 256 65536 8388608 ], start = 0, using indirect IO
mrconvert: [INFO] creating image "DWI.mif"...
mrconvert: [DEBUG] sanitising image information...
mrconvert: [DEBUG] creating empty file "DWI.mif"
mrconvert: [DEBUG] resizing file "DWI.mif" to 1174407324
mrconvert: [DEBUG] sanitising image information...
mrconvert: [DEBUG] memory-mapping file "DWI.mif"...
mrconvert: [DEBUG] "DWI.mif" appears to reside on a networked filesystem - using delayed write-back
mrconvert: [DEBUG] file "DWI.mif" held in RAM at 0x15918f000, size 1174405120
mrconvert: [DEBUG] image "DWI.mif" loaded
mrconvert: [DEBUG] image "DWI.mif" initialised with strides = [ 35 8960 2293760 1 ], start = 0, using indirect IO
mrconvert: [DEBUG] initialising threads...
mrconvert: [DEBUG] launching 8 threads "loop threads"...
mrconvert: [  0%] copying from "before_MR/DWIs_upscaled.nii.gz" to "DWI.mif"...
mrconvert: [DEBUG] waiting for completion of threads "loop threads"...
mrconvert: [100%] copying from "before_MR/DWIs_upscaled.nii.gz" to "DWI.mif"...
mrconvert: [DEBUG] threads "loop threads" completed OK
mrconvert: [100%] copying from "before_MR/DWIs_upscaled.nii.gz" to "DWI.mif"
mrconvert: [INFO] writing back contents of mapped file "DWI.mif"...
libc++abi.dylib: terminating with uncaught exception of type MR::Exception
Abort trap: 6

For the other command (I include -verbose instead of -debug):

dwi2response fa DWI.mif  response_func.txt -verbose
dwi2response: 
dwi2response: Note that this script makes use of commands / algorithms that have relevant articles for citation. Please consult the help page (-help option) for more information.
dwi2response: 
dwi2response: Generated temporary directory: /tmp/dwi2response-tmp-TQOZRL/
Command: mrconvert /<WORKING_DIR>/DWI.mif /tmp/dwi2response-tmp-TQOZRL/dwi.mif -stride 0,0,0,1
dwi2response: 
dwi2response: [ERROR] Command failed: mrconvert /<WORKING_DIR>/DWI.mif /tmp/dwi2response-tmp-TQOZRL/dwi.mif -stride 0,0,0,1
dwi2response: Output of failed command:
mrconvert: [  0%] copying from "/Users/rdp...ata/100902_4TT01167/DWI.mif" to "/tmrconvert: [  1%] copying from "/<WORKING_DIR>/DWI.mif" to "/tmrconvert: [  2%] copying from 
.
.
.
"/<WORKING_DIR>/DWI.mif" to "/tmrconvert: [100%] copying from "/Users/rdp...ata/100902_4TT01167/DWI.mif" to "/tmp/dwi2response-tmp-TQOZRL/dwi.mif"
libc++abi.dylib: terminating with uncaught exception of type MR::Exception
dwi2response: Changing back to original directory (/<WORKING_DIR>/)
dwi2response: Script failed while executing the command: mrconvert /<WORKING_DIR>/DWI.mif /tmp/dwi2response-tmp-TQOZRL/dwi.mif -stride 0,0,0,1
dwi2response: For debugging, inspect contents of temporary directory: /tmp/dwi2response-tmp-TQOZRL/

(In OSX Sierra 10.12.4)
Ok guys,
After reinstalling mrtrix3 and the dependencies using ‘brew install XX’ I tried it again and I keep getting the same libc++abi.dylib error:

Not sure what to do now… :confused:

mrconvert before_MR/DWIs_upscaled.nii.gz DWI.mif -grad mr_btable.txt -datatype float32 -stride 0,0,0,1 -debug 
mrconvert: [DEBUG] No config file found at "/etc/mrtrix.conf"
mrconvert: [DEBUG] No config file found at ".mrtrix.conf"
mrconvert: [INFO] opening image "before_MR/DWIs_upscaled.nii.gz"...
mrconvert: [DEBUG] sanitising image information...
mrconvert: [DEBUG] searching for suitable gradient encoding...
mrconvert: [DEBUG] loading matrix file "mr_btable.txt"...
mrconvert: [DEBUG] found 35x4 matrix in file "mr_btable.txt"
mrconvert: [INFO] found 35x4 diffusion gradient table
mrconvert: [DEBUG] loading image "before_MR/DWIs_upscaled.nii.gz"...
mrconvert: [100%] uncompressing image "before_MR/DWIs_upscaled.nii.gz"
mrconvert: [DEBUG] image "before_MR/DWIs_upscaled.nii.gz" loaded
mrconvert: [DEBUG] image "before_MR/DWIs_upscaled.nii.gz" initialised with strides = [ 1 256 65536 8388608 ], start = 0, using indirect IO
mrconvert: [INFO] creating image "DWI.mif"...
mrconvert: [DEBUG] sanitising image information...
mrconvert: [DEBUG] creating empty file "DWI.mif"
mrconvert: [DEBUG] resizing file "DWI.mif" to 1174407476
mrconvert: [DEBUG] sanitising image information...
mrconvert: [DEBUG] memory-mapping file "DWI.mif"...
mrconvert: [DEBUG] "DWI.mif" appears to reside on a networked filesystem - using delayed write-back
mrconvert: [DEBUG] file "DWI.mif" held in RAM at 0x1613f4000, size 1174405120
mrconvert: [DEBUG] image "DWI.mif" loaded
mrconvert: [DEBUG] image "DWI.mif" initialised with strides = [ 35 8960 2293760 1 ], start = 0, using indirect IO
mrconvert: [DEBUG] initialising threads...
mrconvert: [DEBUG] launching 8 threads "loop threads"...
mrconvert: [  0%] copying from "before_MR/DWIs_upscaled.nii.gz" to "DWI.mif"...
mrconvert: [DEBUG] waiting for completion of threads "loop threads"...
mrconvert: [100%] copying from "before_MR/DWIs_upscaled.nii.gz" to "DWI.mif"...
mrconvert: [DEBUG] threads "loop threads" completed OK
mrconvert: [100%] copying from "before_MR/DWIs_upscaled.nii.gz" to "DWI.mif"
mrconvert: [INFO] writing back contents of mapped file "DWI.mif"...
libc++abi.dylib: terminating with uncaught exception of type MR::Exception
Abort trap: 6

Ok, I think we can get around the uncaught exception error (what you refer to as the libc++abi.dylib error) by disabling multi-threading: add -nthreads 0 to the command. No need to run the dwi2response script, it’s likely to be the same error.

As to what might be going on: given the nature of the original error, I still strongly suspect something odd about the filesystem - which would mean reinstalling won’t help at all. The command I suggested earlier (df -T .) was for Linux, I’m not sure what the equivalent would be for macOS. But I’d expect there would be some utility that would allow you to get the information (is there a disk utility on macOS?). /dev/disk is the hardware device, but this doesn’t tell us what filesystem is used for that partition - I’d expect HFS+ on macOS, but given your debug output, which detected your filesystem as residing on the network when you said this is your internal hard drive, it’s likely to be some other filesystem we haven’t come across yet (MRtrix assumes anything it doesn’t know about is best treated as if it was a network filesystem). I understand Apple’s new APFS may have just been unleashed (?), which might introduce issues all of its own…

What’s odd is that by assuming a network filesystem, the operations used should actually be more widely supported (it’s just one big standard write-back). I don’t understand why this might be falling over…

One last question: can you post the contents of your​ config file? Not sure how easy it will be to get it though, if you’re using homebrew - @maxpietsch might be able to help here. Might be time to try a local install, using these instructions - that way we’ll be able to get the specifics of exactly how everything fits together.

And one more thing: you wouldn’t happen to have Anaconda installed, would you…? That tends to introduce all kinds of exotic issues…

@jdtournier,
I previously installed it using the instructions provided. (I would give it another try).

The information about my filesystem:

Volume name : Macintosh HD
Volume type : Logical Volume
BSD device node : disk1
Mount point : /
File system : Mac OS Extended (Journaled, Encrypted)
Connection : PCI
Device tree path : IODeviceTree:/PCI0@0/PEG0@1/SSD0@0/PRT0@0/PMP@0
Writable : Yes
Is case-sensitive : No
File system UUID : 7EBA2BA9-D067-32C0-9D96-FC2C7CA65F1F
CoreStorage UUID : 24FD0101-129A-41B9-89B2-0622849CC129
Parent CoreStorage LVG UUID : CA9E5E86-7041-4D9B-89F7-34799EACA68E
Volume capacity : 958,809,112,576
Available space (Purgeable + Free) : 723,763,732,612
Purgeable space : 10,105,802,884
Free space : 713,657,929,728
Used space : 235,045,379,964
File count : 2,365,180
Owners enabled : Yes
Is encrypted : Yes
System Integrity Protection supported : Yes
Can be verified : Yes
Can be repaired : No
Bootable : Yes
Journaled : Yes
Disk number : 1
Media name : Macintosh HD
Media type : Generic
Ejectable : Yes
Solid state : No
S.M.A.R.T. status : Not Supported
Parent disks : disk0s2

About anaconda, yes I have it installed :confused:

$which python
/usr/local/anaconda2/bin/python

Now running mrconvert with -nthreads 0 (same as before):

mrconvert before_MR/DWIs_upscaled.nii.gz DWI.mif -grad mr_btable.txt -datatype float32 -stride 0,0,0,1 -nthreads 0 
mrconvert: [100%] uncompressing image "before_MR/DWIs_upscaled.nii.gz"
mrconvert: [100%] copying from "before_MR/DWIs_upscaled.nii.gz" to "DWI.mif"
libc++abi.dylib: terminating with uncaught exception of type MR::Exception
Abort trap: 6

About my config file. Are you refering the to mrtrix config file (check below)? If not, I can get it to you as long as you give me some additional clues on where to find it.

cat /usr/local/Cellar/mrtrix3/0.3.15-1397-g93f159a/share/mrtrix3/config 
#!/usr/bin/python
#
# autogenerated by MRtrix configure script
#
# configure output:
# 
# MRtrix build type requested: release
# 
# Detecting OS: darwin
# OS X deployment target: 10.12
# Looking for compiler [clang++]: Apple LLVM version 8.0.0 (clang-800.0.42.1)
# Checking for C++11 compliance: ok
# Checking for ::max_align_t: ok
# Checking for std::max_align_t: ok
# Detecting pointer size: 64 bit
# Detecting byte order: little-endian
# Checking for variable-length array support: yes
# Checking for non-POD variable-length array support: no
# Checking for zlib compression library: 1.2.8
# Checking for Eigen 3 library: 3.2.6
# Checking JSON for Modern C++ requirements: OK
# Checking shared library generation: yes
# Checking for Qt moc: moc (version 5.6.0)
# Checking for Qt qmake: qmake (version 5.6.0)
# Checking for Qt rcc: rcc (version 5.6.0)
# Checking for Qt: 5.6.0


PATH = r'/usr/local/anaconda2/bin:/usr/local/Library/ENV/4.3:/usr/local/opt/pkg-config/bin:/usr/bin:/bin:/usr/sbin:/sbin:/usr/local/anaconda2/bin'
obj_suffix = '.o'
exe_suffix = ''
lib_prefix = 'lib'
lib_suffix = '.dylib'
cpp = [ 'clang++', '-c', 'CFLAGS', 'SRC', '-o', 'OBJECT' ]
cpp_flags = [ '-std=c++11', '-DMRTRIX_MACOSX', '-fPIC', '-mmacosx-version-min=10.12', '-DMRTRIX_WORD64', '-DMRTRIX_NO_NON_POD_VLA', '-Wall', '-O3', '-DNDEBUG' ]
ld = [ 'clang++', 'OBJECTS', 'LDFLAGS', '-o', 'EXECUTABLE' ]
ld_flags = [ '-mmacosx-version-min=10.12', '-lz' ]
runpath = '-Wl,-rpath,@loader_path/'
ld_enabled = True
ld_lib = [ 'clang++', 'OBJECTS', 'LDLIB_FLAGS', '-o', 'LIB' ]
ld_lib_flags = [ '-dynamiclib', '-install_name', '@rpath/LIBNAME', '-mmacosx-version-min=10.12', '-mmacosx-version-min=10.12', '-lz' ]
eigen_cflags = [ '-isystem', '/usr/local/Cellar/eigen/3.2.6/include/eigen3', '-DEIGEN_DONT_PARALLELIZE' ]
moc = 'moc'
rcc = 'rcc'
qt_cflags = [ '-pipe', '-stdlib=libc++', '-std=gnu++11', '-isysroot', '/Applications/Xcode.app/Contents/Developer/Platforms/MacOSX.platform/Developer/SDKs/MacOSX10.12.sdk', '-mmacosx-version-min=10.12', '-fPIC', '-DQT_NO_DEBUG', '-DQT_OPENGL_LIB', '-DQT_SVG_LIB', '-DQT_WIDGETS_LIB', '-DQT_GUI_LIB', '-DQT_CORE_LIB', '-isystem', '/usr/local/anaconda2/include/qt', '-isystem', '/usr/local/anaconda2/include/qt/QtOpenGL', '-isystem', '/usr/local/anaconda2/include/qt/QtSvg', '-isystem', '/usr/local/anaconda2/include/qt/QtWidgets', '-isystem', '/usr/local/anaconda2/include/qt/QtGui', '-isystem', '/usr/local/anaconda2/include/qt/QtCore', '-isystem', '/Applications/Xcode.app/Contents/Developer/Platforms/MacOSX.platform/Developer/SDKs/MacOSX10.12.sdk/System/Library/Frameworks/OpenGL.framework/Headers', '-isystem', '/Applications/Xcode.app/Contents/Developer/Platforms/MacOSX.platform/Developer/SDKs/MacOSX10.12.sdk/System/Library/Frameworks/AGL.framework/Headers', '-isystem', '/usr/local/anaconda2/mkspecs/macx-clang' ]
qt_ldflags = [ '-headerpad_max_install_names', '-stdlib=libc++', '-Wl,-syslibroot,/Applications/Xcode.app/Contents/Developer/Platforms/MacOSX.platform/Developer/SDKs/MacOSX10.12.sdk', '-mmacosx-version-min=10.12', '-Wl,-rpath,/usr/local/anaconda2/lib', '-L/usr/local/anaconda2/lib', '-lQt5OpenGL', '-lQt5Svg', '-lQt5Widgets', '-lQt5Gui', '-lQt5Core', '-framework', 'OpenGL', '-framework', 'AGL' ]
nogui = False

((**Also, I have to “\#” all the # chararcters or else they would not appear and will make the text bold. Is there an easier way for future references?))

Fixed the formatting for you in your previous posts. You can enclose code within backticks - single backticks for inline code, triple backticks on a single line for blocks of code. Try editing your previous post, you’ll see how I modified it.

Back to your issue:

Yes, that’s HFS+. No idea why it’s detected as a network filesystem… On that note, I got it wrong in my previous post: the tests happen the other way around, MRtrix3 checks for known network filesystems, and assumes local otherwise - in case this confuses anyone…

OK, so this probably means that the exception is thrown from a destructor. I’ll have to look into why the error message isn’t displaying… But that’s a separate issue, the error shouldn’t happen at all.

More to the point, it’s not unlikely that Anaconda is causing these issues:

It’s clearly in your PATH during the build - I’m not saying that it definitely is the cause of the problem, but Anaconda causes enough trouble that it’s worth trying to build MRtrix3 without it interfering. Could you try removing all mentions of Anaconda from your PATH while installing MRtrix3, see if that fixes it? Something like this might work:

export PATH=/usr/local/Library/ENV/4.3:/usr/local/opt/pkg-config/bin:/usr/bin:/bin:/usr/sbin:/sbin
brew install mrtrix3

I’m hoping that’ll be sufficient - you should be able to check whether your export PATH directive was honoured after the install by looking at the same line in the config - it should be the same as you provided. If for some reason the PATH still reflects what you had before, that’ll probably because HomeBrew re-loads your shell’s startup file - in which case you’ll need to find a way to remove all mention of Anaconda from the PATH using a more permanent solution (e.g. editing your ~/.profile file - I assume that’s where it would be set). You can always set it back to what it was after the MRtrix3 install, it’s only necessary during the build process.

Failing that, you could always install from source, as per the instructions - you’ve already got all the dependencies installed, it should actually be a trivial matter of:

export PATH=/usr/local/Library/ENV/4.3:/usr/local/opt/pkg-config/bin:/usr/bin:/bin:/usr/sbin:/sbin
git clone https://github.com/MRtrix3/mrtrix3.git
cd mrtrix3
./configure
./build
./set_path

Might prove easier if Anaconda is still interfering with the HomeBrew install…

@Drigo We updated the configure script. Could you please try to reinstall mrtrix and see whether it solves the issue?

Just to clarify: the changes we just made should sort out conflicts due to interference from Anaconda, whether installing manually or via brew install mrtrix3. It would be great to get some feedback as to whether this does actually fix these odd errors you were getting…

Ok, so I tried it with brew install mrtrix3 after doing this:

$ export PATH=/usr/local/bin:/usr/bin:/bin:/usr/sbin:/sbin:/opt/X11/bin
brew install mrtrix3

and then I applied (the same which doesn’t fix the problem):

$ mrconvert before_MR/DWIs_upscaled.nii.gz DWI.mif -grad mr_btable.txt -datatype float32 -stride 0,0,0,1 -debug
mrconvert: [DEBUG] No config file found at "/etc/mrtrix.conf"
mrconvert: [DEBUG] No config file found at ".mrtrix.conf"
mrconvert: [INFO] opening image "before_MR/DWIs_upscaled.nii.gz"...
mrconvert: [DEBUG] sanitising image information...
mrconvert: [DEBUG] searching for suitable gradient encoding...
mrconvert: [DEBUG] loading matrix file "mr_btable.txt"...
mrconvert: [DEBUG] found 35x4 matrix in file "mr_btable.txt"
mrconvert: [INFO] found 35x4 diffusion gradient table
mrconvert: [DEBUG] loading image "before_MR/DWIs_upscaled.nii.gz"...
mrconvert: [100%] uncompressing image "before_MR/DWIs_upscaled.nii.gz"
mrconvert: [DEBUG] image "before_MR/DWIs_upscaled.nii.gz" loaded
mrconvert: [DEBUG] image "before_MR/DWIs_upscaled.nii.gz" initialised with strides = [ 1 256 65536 8388608 ], start = 0, using indirect IO
mrconvert: [INFO] creating image "DWI.mif"...
mrconvert: [DEBUG] sanitising image information...
mrconvert: [DEBUG] creating empty file "DWI.mif"
mrconvert: [DEBUG] resizing file "DWI.mif" to 1174407468
mrconvert: [DEBUG] sanitising image information...
mrconvert: [DEBUG] memory-mapping file "DWI.mif"...
mrconvert: [DEBUG] "DWI.mif" appears to reside on a networked filesystem - using delayed write-back
mrconvert: [DEBUG] file "DWI.mif" held in RAM at 0x159b6e000, size 1174405120
mrconvert: [DEBUG] image "DWI.mif" loaded
mrconvert: [DEBUG] image "DWI.mif" initialised with strides = [ 35 8960 2293760 1 ], start = 0, using indirect IO
mrconvert: [DEBUG] initialising threads...
mrconvert: [DEBUG] launching 8 threads "loop threads"...
mrconvert: [  0%] copying from "before_MR/DWIs_upscaled.nii.gz" to "DWI.mif"... 
mrconvert: [DEBUG] waiting for completion of threads "loop threads"...
mrconvert: [100%] copying from "before_MR/DWIs_upscaled.nii.gz" to "DWI.mif"... 
mrconvert: [DEBUG] threads "loop threads" completed OK
mrconvert: [100%] copying from "before_MR/DWIs_upscaled.nii.gz" to "DWI.mif"
mrconvert: [INFO] writing back contents of mapped file "DWI.mif"...
libc++abi.dylib: terminating with uncaught exception of type MR::Exception
Abort trap: 6

Then I tried the github installation:

$ export PATH=/usr/local/Library/ENV/4.3:/usr/local/opt/pkg-config/bin:/usr/bin:/bin:/usr/sbin:/sbin:/usr/local/Cellar/qt/5.8.0_2/bin
$ git clone https://github.com/MRtrix3/mrtrix3.git
cd mrtrix3
./configure
./build
./set_path  #This gave me some permission issues (check below) but I was able to replicate it since I believe it only adds to path)

and then my mrconvert command ( ) gives me the same error :

$ mrconvert before_MR/DWIs_upscaled.nii.gz DWI.mif -grad mr_btable.txt -datatype float32 -stride 0,0,0,1 -debug 
mrconvert: [DEBUG] No config file found at "/etc/mrtrix.conf"
mrconvert: [DEBUG] No config file found at ".mrtrix.conf"
mrconvert: [INFO] opening image "before_MR/DWIs_upscaled.nii.gz"...
mrconvert: [DEBUG] sanitising image information...
mrconvert: [DEBUG] searching for suitable gradient encoding...
mrconvert: [DEBUG] loading matrix file "mr_btable.txt"...
mrconvert: [DEBUG] found 35x4 matrix in file "mr_btable.txt"
mrconvert: [INFO] found 35x4 diffusion gradient table
mrconvert: [DEBUG] loading image "before_MR/DWIs_upscaled.nii.gz"...
mrconvert: [100%] uncompressing image "before_MR/DWIs_upscaled.nii.gz"
mrconvert: [DEBUG] image "before_MR/DWIs_upscaled.nii.gz" loaded
mrconvert: [DEBUG] image "before_MR/DWIs_upscaled.nii.gz" initialised with strides = [ 1 256 65536 8388608 ], start = 0, using indirect IO
mrconvert: [INFO] creating image "DWI.mif"...
mrconvert: [DEBUG] sanitising image information...
mrconvert: [DEBUG] creating empty file "DWI.mif"
mrconvert: [DEBUG] resizing file "DWI.mif" to 1174407488
mrconvert: [DEBUG] sanitising image information...
mrconvert: [DEBUG] memory-mapping file "DWI.mif"...
mrconvert: [DEBUG] "DWI.mif" appears to reside on a networked filesystem - using delayed write-back
mrconvert: [DEBUG] file "DWI.mif" held in RAM at 0x1544a9000, size 1174405120
mrconvert: [DEBUG] image "DWI.mif" loaded
mrconvert: [DEBUG] image "DWI.mif" initialised with strides = [ 35 8960 2293760 1 ], start = 0, using indirect IO
mrconvert: [DEBUG] initialising threads...
mrconvert: [DEBUG] launching 8 threads "loop threads"...
mrconvert: [  0%] copying from "before_MR/DWIs_upscaled.nii.gz" to "DWI.mif"... 
mrconvert: [DEBUG] waiting for completion of threads "loop threads"...
mrconvert: [100%] copying from "before_MR/DWIs_upscaled.nii.gz" to "DWI.mif"... 
mrconvert: [DEBUG] threads "loop threads" completed OK
mrconvert: [100%] copying from "before_MR/DWIs_upscaled.nii.gz" to "DWI.mif"
mrconvert: [INFO] writing back contents of mapped file "DWI.mif"...
ffflibc++abi.dylib: terminating with uncaught exception of type MR::Exception
Abort trap: 6 

**Permission issues when executing ./set_path (this information may be helpful??):

./set_path 
Traceback (most recent call last):
  File "./set_path", line 105, in <module>
    with open (filename, 'w') as f:
IOError: [Errno 13] Permission denied: '/Users/rdp20/.bash_profile'

OK, I guess this rules out Anaconda as the cause of the problem - thanks for all the testing.

However, the failure in ./set_path is in keeping with your first reported error, and is completely unexpected. This is a simple python script, trying to open a file in your account for writing, in a completely routine way - and that fails with permission denied? The only reasons I can think of are:

  • the permissions on your account are genuinely badly set. What does ls -l ~/.bash_profile report? What does ls -l ~ report? You need to double-check that you are the owner of all the files in your account, and that you do have the right permissions set to actually access them. I’ve seen issues like this after an upgrade or re-install, where your user ID changes during the process and no longer matches with the user ID of the files stored in your account. Or maybe as a result of operating as root while manipulating files in your account, which can potentially result in these files being owned by the root account rather than your own…

  • you have some over-zealous anti-virus / anti-malware / other security policy set up on your system, which blocks any non-trusted application from doing anything damaging / useful. I’ve no idea how these things work on macOS, unfortunately…

Thank you for all your help. At this moment, I am not sure how to fix it.
Instead of using ~/.bash_profile, I use ~/.bashrc and both are owned by me:

ls -l ~/.bashrc
-rw-r–r--@ 1 rdp20 staff 4993 May 3 19:00 /Users/rdp20/.bashrc
rdp20: 100902_4TT01167$ls -l ~/.bash_profile
-rw-r–r-- 1 rdp20 staff 210 May 4 22:38 /Users/rdp20/.bash_profile

I am not sure what to do now but thanks for your help.
Rodrigo

Could somebody help me understand how is this library (libc++abi.dylib ) being called so I can meticously check what may be going on?

OK, my guess is that the libc++abi.dylib message is a touch misleading. What matters here is the uncaught exception. Normally, exceptions should propagate all the way to the top-level function in the app (main()), and be handled there. The only times this doesn’t happen is when the exception is throw from one of the threads in multi-threaded app (not the case, since you get this error with -nthread 0 anyway), or when invoked from a class destructor not explicitly marked as noexcept (false). The problem is that the most likely place this exception is being thrown from is the File::MMap destructor where the image data write-back is handled, particularly given your earlier ‘permission denied’ errors. However, that destructor is marked noexcept (false), so I don’t understand why this exception wouldn’t be caught as expected - unless once again Apple have decided to go against the C++ standard (wouldn’t be the first time…).

If you really want to debug this properly, this is what I recommend:

  • build a debug version of the code:

    • stash your current executables and create a fresh config:

        $ ./build select debug
      
    • configure a full debug build:

       $ ./configure -assert -debug
      
    • compile just the mrconvert executable to speed things up:

       $ ./build bin/mrconvert
      
  • run the command within the LLVM debugger:

      $ lldb --args bin/mrconvert before_MR/DWIs_upscaled.nii.gz DWI.mif -grad mr_btable.txt -datatype float32 -stride 0,0,0,1 -debug
    

    then type r at the prompt, wait for the crash, and then type bt to dump the backtrace - the full function stack at that point. Somewhere in there will be some clear information as to where exactly the error occurred… Feel free to copy/paste the full terminal output here so we can help figure this out (use the </> button to format that section as a verbatim/code block).

Once you’re done, you can revert back to your original config/build using this command:

$ ./build select default

Hope this helps…

Just thought I’d add a further 2 cents while I’m at it… I still think the issue has to do with your system blocking file access for some reason. The error you get with mrconvert was originally a ‘permission denied’ issue, and the latest libc++abi.dylib (uncaught exception) error is actually still compatible with that, given that it’s essentially saying “an error occurred that wasn’t dealt with”, and the line immediately preceding this error (when running with the -debug flag) is immediately before the file write operation. So it’s still very likely that the write failed and this is the error we now see.

So I had a quick look into what kind of antivirus or other system protection might be in place on macOS, and came across GateKeeper. It blocks any non-trusted application from running - and we (the MRtrix3 team) aren’t (yet) trusted… Apparently this was available in previous versions of macOS, but not enabled by default - but it is now on by default in Sierra. I’m not convinced this is the problem here, since the command is blocked when it tried to write data, rather than being prevented from running at all. But it might be worth looking into the settings for GateKeeper to see if there’s a chance that could be the issue. Maybe you could disable it briefly just to see if that will allow your command to complete…?

  1. GateKeeper doesn’t seem to be a problem or Sierra in OSX as I had a coworker install mrtrix3 in his OSX Sierra MacBook with no problems following the Github installation instructions. Apparently, the problem is something regarding my computer.

  2. Anyhow, after doing the following instructions below:

$ lldb -- bin/mrconvert DWIs_upscaled.nii.gz DWI.mif -grad mr_btable.txt -datatype float32 -stride 0,0,0,1 -debug
(lldb) target create "bin/mrconvert"
Current executable set to 'bin/mrconvert' (x86_64).
(lldb) settings set -- target.run-args  "DWIs_upscaled.nii.gz" "DWI.mif" "-grad" "mr_btable.txt" "-datatype" "float32" "-stride" "0,0,0,1" "-debug"
(lldb)

Then I typed ‘r’:

mrconvert: [DEBUG] No config file found at "/etc/mrtrix.conf"
mrconvert: [DEBUG] No config file found at ".mrtrix.conf"
mrconvert: [INFO] opening image "DWIs_upscaled.nii.gz"...
mrconvert: [DEBUG] sanitising image information...
mrconvert: [DEBUG] searching for suitable gradient encoding...
mrconvert: [DEBUG] loading matrix file "mr_btable.txt"...
mrconvert: [DEBUG] found 35x4 matrix in file "mr_btable.txt"
mrconvert: [INFO] found 35x4 diffusion gradient table
mrconvert: [DEBUG] loading image "DWIs_upscaled.nii.gz"...
mrconvert: [100%] uncompressing image "DWIs_upscaled.nii.gz"
mrconvert: [DEBUG] image "DWIs_upscaled.nii.gz" loaded
mrconvert: [DEBUG] image "DWIs_upscaled.nii.gz" initialised with strides = [ 1 256 65536 8388608 ], start = 0, using indirect IO
mrconvert: [INFO] creating image "DWI.mif"...
mrconvert: [DEBUG] sanitising image information...
mrconvert: [DEBUG] creating empty file "DWI.mif"
mrconvert: [DEBUG] resizing file "DWI.mif" to 1174407480
mrconvert: [DEBUG] sanitising image information...
mrconvert: [DEBUG] memory-mapping file "DWI.mif"...
mrconvert: [DEBUG] "DWI.mif" appears to reside on a networked filesystem - using delayed write-back
mrconvert: [DEBUG] file "DWI.mif" held in RAM at 0x149000000, size 1174405120
mrconvert: [DEBUG] image "DWI.mif" loaded
mrconvert: [DEBUG] image "DWI.mif" initialised with strides = [ 35 8960 2293760 1 ], start = 0, using indirect IO
mrconvert: [DEBUG] initialising threads...
mrconvert: [DEBUG] launching 8 threads "loop threads"...
mrconvert: [  0%] copying from "DWIs_upscaled.nii.gz" to "DWI.mif"... 
mrconvert: [DEBUG] waiting for completion of threads "loop threads"...
mrconvert: [100%] copying from "DWIs_upscaled.nii.gz" to "DWI.mif"... 
mrconvert: [DEBUG] threads "loop threads" completed OK
mrconvert: [100%] copying from "DWIs_upscaled.nii.gz" to "DWI.mif"
mrconvert: [INFO] writing back contents of mapped file "DWI.mif"...
libc++abi.dylib: terminating with uncaught exception of type MR::Exception
Process 50259 stopped`
 thread #1: tid = 0x4ebf22, 0x00007fffc4387d42 libsystem_kernel.dylib`__pthread_kill + 10, queue = 'com.apple.main-thread', stop reason = signal SIGABRT
    frame #0: 0x00007fffc4387d42 libsystem_kernel.dylib`__pthread_kill + 10
libsystem_kernel.dylib`__pthread_kill:
->  0x7fffc4387d42 <+10>: jae    0x7fffc4387d4c            ; <+20>
    0x7fffc4387d44 <+12>: movq   %rax, %rdi
    0x7fffc4387d47 <+15>: jmp    0x7fffc4380caf            ; cerror_nocancel
    0x7fffc4387d4c <+20>: retq   
(lldb)

Then after typing ‘bt’:

* thread #1: tid = 0x4ebf22, 0x00007fffc4387d42 libsystem_kernel.dylib`__pthread_kill + 10, queue = 'com.apple.main-thread', stop reason = signal SIGABRT
  * frame #0: 0x00007fffc4387d42 libsystem_kernel.dylib`__pthread_kill + 10
    frame #1: 0x00007fffc44755bf libsystem_pthread.dylib`pthread_kill + 90
    frame #2: 0x00007fffc42ed420 libsystem_c.dylib`abort + 129
    frame #3: 0x00007fffc2e4384a libc++abi.dylib`abort_message + 266
    frame #4: 0x00007fffc2e68c4f libc++abi.dylib`default_terminate_handler() + 267
    frame #5: 0x00007fffc3976713 libobjc.A.dylib`_objc_terminate() + 124
    frame #6: 0x00007fffc2e65d69 libc++abi.dylib`std::__terminate(void (*)()) + 8
    frame #7: 0x00007fffc2e65de3 libc++abi.dylib`std::terminate() + 51
    frame #8: 0x00000001003887eb libmrtrix.dylib`__clang_call_terminate + 11
    frame #9: 0x00000001004e797c libmrtrix.dylib`std::__1::__shared_ptr_pointer<MR::File::MMap*, std::__1::default_delete<MR::File::MMap>, std::__1::allocator<MR::File::MMap> >::__on_zero_shared() [inlined] std::__1::default_delete<MR::File::MMap>::operator(__ptr=<unavailable>)(MR::File::MMap*) const + 60 at memory:2525 [opt]
    frame #10: 0x00000001004e796c libmrtrix.dylib`std::__1::__shared_ptr_pointer<MR::File::MMap*, std::__1::default_delete<MR::File::MMap>, std::__1::allocator<MR::File::MMap> >::__on_zero_shared(this=<unavailable>) + 44 at memory:3768 [opt]
    frame #11: 0x00007fffc2e26dae libc++.1.dylib`std::__1::__shared_weak_count::__release_shared() + 44
    frame #12: 0x00000001004e73bd libmrtrix.dylib`MR::ImageIO::Default::unload(MR::Header const&) [inlined] std::__1::allocator<std::__1::shared_ptr<MR::File::MMap> >::destroy(__p=<unavailable>) + 141 at memory:1795 [opt]
    frame #13: 0x00000001004e73b8 libmrtrix.dylib`MR::ImageIO::Default::unload(MR::Header const&) [inlined] void std::__1::allocator_traits<std::__1::allocator<std::__1::shared_ptr<MR::File::MMap> > >::__destroy<std::__1::shared_ptr<MR::File::MMap> >(__p=<unavailable>) at memory:1668 [opt]
    frame #14: 0x00000001004e73b8 libmrtrix.dylib`MR::ImageIO::Default::unload(MR::Header const&) [inlined] void std::__1::allocator_traits<std::__1::allocator<std::__1::shared_ptr<MR::File::MMap> > >::destroy<std::__1::shared_ptr<MR::File::MMap> >(__p=<unavailable>) at memory:1536 [opt]
    frame #15: 0x00000001004e73b8 libmrtrix.dylib`MR::ImageIO::Default::unload(MR::Header const&) [inlined] std::__1::__vector_base<std::__1::shared_ptr<MR::File::MMap>, std::__1::allocator<std::__1::shared_ptr<MR::File::MMap> > >::__destruct_at_end(__new_last=0x0000000100e002e0) + 8 at vector:424 [opt]
    frame #16: 0x00000001004e73b0 libmrtrix.dylib`MR::ImageIO::Default::unload(MR::Header const&) [inlined] std::__1::__vector_base<std::__1::shared_ptr<MR::File::MMap>, std::__1::allocator<std::__1::shared_ptr<MR::File::MMap> > >::clear() + 16 at vector:368 [opt]
    frame #17: 0x00000001004e73a0 libmrtrix.dylib`MR::ImageIO::Default::unload(MR::Header const&) [inlined] std::__1::vector<std::__1::shared_ptr<MR::File::MMap>, std::__1::allocator<std::__1::shared_ptr<MR::File::MMap> > >::clear() at vector:740 [opt]
    frame #18: 0x00000001004e73a0 libmrtrix.dylib`MR::ImageIO::Default::unload(this=<unavailable>, header=<unavailable>) + 112 at default.cpp:71 [opt]
    frame #19: 0x00000001004c84c2 libmrtrix.dylib`MR::ImageIO::Base::close(this=0x0000000100e02ad0, header=0x0000000100f03cc0) + 50 at base.cpp:51 [opt]
    frame #20: 0x000000010000b4fe mrconvert`MR::Header::~Header(this=0x0000000100f03cc0) + 30 at header.h:176 [opt]
    frame #21: 0x000000010001c3c7 mrconvert`std::__1::__shared_ptr_pointer<MR::Image<int>::Buffer*, std::__1::default_delete<MR::Image<int>::Buffer>, std::__1::allocator<MR::Image<int>::Buffer> >::__on_zero_shared() [inlined] std::__1::default_delete<MR::Image<int>::Buffer>::operator(__ptr=0x0000000100f03cc0)(MR::Image<int>::Buffer*) const + 13 at memory:2525 [opt]
    frame #22: 0x000000010001c3ba mrconvert`std::__1::__shared_ptr_pointer<MR::Image<int>::Buffer*, std::__1::default_delete<MR::Image<int>::Buffer>, std::__1::allocator<MR::Image<int>::Buffer> >::__on_zero_shared(this=<unavailable>) + 10 at memory:3768 [opt]
    frame #23: 0x00007fffc2e26dae libc++.1.dylib`std::__1::__shared_weak_count::__release_shared() + 44
    frame #24: 0x000000010006026f mrconvert`MR::Image<double>::~Image(this=0x00007fff5fbfe598) + 447 at image.h:399 [opt]
    frame #25: 0x0000000100007373 mrconvert`void copy_permute<double>(header_in=<unavailable>, header_out=0x00007fff5fbff780, pos=0x00007fff5fbff398, output_filename="DWI.mif") + 579 at mrconvert.cpp:233 [opt]
    frame #26: 0x00000001000031a1 mrconvert`run() + 5265 at mrconvert.cpp:397 [opt]
    frame #27: 0x0000000100000d7d mrconvert`main(cmdline_argc=<unavailable>, cmdline_argv=<unavailable>) + 45 at command.h:89 [opt]
    frame #28: 0x00007fffc4259235 libdyld.dylib`start + 1

I just happened to have a chat with my boss (he has a shiny new Macbook Pro), and apparently the only issues he’s had with it so far has been antivirus-related… IT loaded the Microsoft System Center Endpoint Protection for Mac. It seems to include a Real-time file system protection feature, which was what was causing trouble - and this sounds eminently capable of causing the issues you’ve been reporting. On his system, it showed up as an icon in the top bar next to the network icon - worth checking whether you have anything like that installed…

Regarding your debug output, this seems to be the relevant entry in the backtrace:

which points to the bit of code responsible for deleting the File::MMap - i.e. invoking its destructor - this is precisely where I was pointing to in my previous post - the issue is most likely the same as your previous reports suggest: permission denied.

However, on closer inspection of the C++11 documentation, it does look like std::shared_ptr expects its contents to have a non-throwing destructor - so it looks like there is an issue with how we handle these errors within MRtrix3, which I’ll fix when I have a minute. But all that would do is catch and report the appropriate error, it won’t prevent the error from occurring in the first place…

So to cut a long story short: it still looks like something on your system is interfering with MRtrix3 in some way…

Thank you all for your help. I now decided to use another machine (now linux CentOS) to run mrtrix and so far I am passed that problem.

I am now running the 5ttgen (w/ fsl 5.0.9) in my T1 image:
$5ttgen fsl T1_brain.nii.gz 5tt_out -nocleanup

I was wondering how long will this process generally takes?