The MRtrix3 Python script library

rsmith · February 8, 2019, 3:51am

In addition to the principal binary commands in MRtrix3, which are written in the C++ language, MRtrix3 also now includes a number of Python scripts for performing common image processing tasks that can be achieved through a combination of existing commands. These make use of a relatively simple Python module library, which provide a certain level of convenience and consistency for building such scripts (e.g. help pages formatted identically to the MRtrix3 binary commands; command-line parsing; creation, use and deletion of temporary scratch directory; control over command-line verbosity; various convenience functions).

It is hoped that in addition to growing in complexity and capability over time, this library may also be of assistance to users when building their own processing scripts, rather than the use of e.g. Bash. It is typically easiest to commence construction of such a script by making a copy of an existing functional script from the MRtrix3 bin/ directory, and then editing the contents to suit your needs.

Executing a script that uses the MRtrix3 libraries

If you wish to be able to use such a script (either written by yourself, or perhaps provided to you externally from the main MRtrix3 package), there are two options that you must choose from in order for execution of that script to be possible:

Place the script file within the bin/ directory of your MRtrix3 installation. The same mechanism that is used within those Python scripts provided with MRtrix3 for locating and loading the MRtrix3 Python libraries will be invoked for this script also, as long as the relevant code is present (see point 2.1 below).
If you wish to be able to place the executable Python script anywhere on your file system, there are two additional steps required:
1. In the Python scripts provided with MRtrix3, you will consistently find the following code snippet:
```
# Make the corresponding MRtrix3 Python libraries available
import inspect, os, sys
LIB_FOLDER = os.path.realpath(os.path.join(os.path.dirname(os.path.realpath(inspect.getfile(inspect.currentframe()))), os.pardir, 'lib'))
if not os.path.isdir(LIB_FOLDER):
  sys.stderr.write('Unable to locate MRtrix3 Python libraries')
  sys.exit(1)
sys.path.insert(0, LIB_FOLDER)
```
  You will need to delete this piece of code from your script file. This code allows Python to locate and load the MRtrix3 Python libraries based on their location on the file system relative to the script currently being executed, which is assumed to be in the MRtrix3 bin/ directory. Since this is not the case for option 2 described here, it needs to be removed.
2. Since Python is not able to locate these libraries automatically in this usage scenario, it is necessary for you to manually provide Python with the information regarding their whereabouts. This can be done using the environment variable PYTHONPATH:
```
$ export PYTHONPATH=/home/user/mrtrix3/lib:$PYTHONPATH
$ ./my_script [arguments] (options)
```
  (Replace the path to the MRtrix3 “lib” directory in the example above with the location of your own installation)

Writing a script using the MRtrix3 libraries

For a small example script please see the demo presented at ISMRM 2020 of the “C++ / Python API & module system”: OSF | commands.txt. Please also see the documentation External modules — MRtrix 3.0 documentation and corresponding publication https://DOI:10.1016/j.neuroimage.2019.116137.

More details to come if there is sufficient demand for it…

Hilikus · October 26, 2021, 12:23am

What does this python library provide exactly?
I’m trying to write a python script as part of a processing pipeline and this script needs to invoke some mrtrix actions. Is there a python API to invoke any action that mrtrix CLI can do? i’m trying to avoid having to spawn a new process with each mrtrix command and having to parse its human-readable stdout output. Is this what the python library is supposed to do? can i access all of mrtrix functionality with it? part of it?

thank you

rsmith · October 27, 2021, 1:29am

Hi @Hilikus,

The MRtrix3 Python API is a fairly lightweight wrapper for automating image processing operations that can be done using a combination of existing commands. I think it’s best thought of as a better alternative to concatenating a set of commands in a Bash script. It’s about daisy-chaining commands to serve some higher-order purpose, but within a framework that intrinsically provides some useful benefits (e.g. standardised interface, command-line parsing, library functions that have proven useful to me over the years) for minimal additional development investment. But it’s explicitly not designed around direct manipulation of raw image data within Python, and it doesn’t have the internal complexity of pipeline construction of something like nipype.

Given that the majority of MRtrix3 commands are C++ compiled binaries rather than Python, spawning processes for each command executed is unavoidable. The API is also not constrained to execute MRtrix3 commands only; there are some nuances that apply only to MRtrix3 commands, but for the most part it simply uses the subprocess module to execute commands that exist in PATH.

Cheers
Rob

Hilikus · November 2, 2021, 4:24pm

thank you for the info @rsmith

This sounds very useful. I know that internally the library must spawn processes but what i meant is along the lines of what you said, a better alternative of me spawning processes directly a la bash script. Is there any sample somewhere where I can see how to use it? i can’t find anything on how to use it and what functions are available in the python library

Thank you

maxpietsch · November 2, 2021, 5:22pm

@Hilikus, I’ve added a few links to The MRtrix3 Python script library that you might find useful.

Hilikus · November 2, 2021, 7:45pm

Thank you @maxpietsch .
I’m still stuck with your sample. I think the important part (the actual python sample) is this one, right? I try to reproduce this in my script but I have this error

Cannot find reference ‘app’ in ‘mrtrix3.py’

Even before I tried your method, i was (i think) able to load the mrtrix3.py module into my python script by following the instructions here
However, like I mentioned above, the mrtrix3 module doesn’t contain a lot of the attributes that you reference in your script, like app, image, path, or run. I’m not sure if that means that the import is not working properly. as far as I can debug it, it is returning success. In my mrtrix3 modules I only see

Unfortunately, running your bash script also failed when it tried to compile mrtrix after cloning it. It complains about Eigen3 in ./configure -noshared -nogui and I don’t want to go down the route of recompiling mrtrix, i’m sure that the precompiled version can still be used in python, right?

So I can think of two things

My import is not working properly and it’s not bringing everything that the mrtrix python module has to offer and that your sample mralign.py is using
your sample script was importing a different version of the mrtrix3 python module that has changed now but because you are checking out master in your bash script, it doesn’t work any more

thanks for any pointers. and while at it, is there any documentaiton of what’s inside the mrtrix python module? how did you know about mrtrix3.app, mrtrix3.run, etc

maxpietsch · November 2, 2021, 10:59pm

Looks like your setup of the module did not work as it is importing and showing the attributes of bin/mrtrix3.py rather than of the lib/mrtrix3/ module located in the main mrtrix3 installation folder.

If you want to tinker yourself, you can use the following test script (mrtest) and compare your output to mine. Otherwise, I’d suggest opening a new (non-wiki) follow up post with full details about the installation method and setup of the module so we can reproduce it. I think external python scripts should work with the binary installers (and the Docker image) but I suspect that’s not widely tested so details about your installation would be helpful.

Documentation is fairly thin but as Rob said, it’s a lightweight module. Anything that can be done in bash can be done in the Python API as well – plus some convenience features such as a scratch directory and parsing header information… as listed above. I’d suggest you start with the example script and go copy-pasting from the shorter scripts in mrtrix3/bin at master · MRtrix3/mrtrix3 · GitHub. You’ll get to know 95% of the functionality you need fairly quickly (and if you’re hungry for more: MRtrix3_connectome/mrtrix3_connectome.py at master · BIDS-Apps/MRtrix3_connectome · GitHub).

Debug script:

#!/usr/bin/env python

def usage(cmdline):
  cmdline.set_author('Author')
  cmdline.set_synopsis('Test installation')
  cmdline.add_argument('dummy', help='not used')
  cmdline.add_description('Check module.')


def execute():
  import inspect
  print('module location:')
  print(mrtrix3.__file__)

  print('module attributes:')
  print(dir(mrtrix3))

  src = inspect.getsource(mrtrix3)
  print('module source:')
  print(src)

  from mrtrix3 import MRtrixError, app, image, path, run

# Execute the script
import mrtrix3
mrtrix3.execute()

which is in my case located in ~/mrtrix3_extra/mralign/bin:

➜ tree mralign
mralign
├── bin
│   ├── mralign
│   ├── mrtest
│   ├── mrtrix3.py -> ../../mrtrix3/bin/mrtrix3.py
│   └── mrtrix3.pyc
├── build
└── cmd

➜  mrtrix3_extra cat mralign/build
/Users/mp/mrtrix3_extra/mrtrix3/build

produces the output

➜  mrtest 0
module location:
/Users/mp/mrtrix3_extra/mrtrix3/lib/mrtrix3/__init__.py
module attributes:
['ANSI', 'ANSICodes', 'BIN_PATH', 'COMMAND_HISTORY_STRING', 'CONFIG', 'EXE_LIST', 'MRtrixBaseError', 'MRtrixError', '__builtins__', '__cached__', '__doc__', '__file__', '__loader__', '__name__', '__package__', '__path__', '__spec__', '__version__', '__warningregistry__', '_version', 'app', 'arg', 'build_path', 'config_path', 'execute', 'f', 'find_executable', 'fp', 'imp', 'imported', 'inspect', 'line', 'namedtuple', 'os', 'quote', 'run', 'setup_ansi', 'sys', 'utils']
module source:
# Copyright (c) 2008-2021 the MRtrix3 contributors.
#
# This Source Code Form is subject to the terms of the Mozilla Public
# License, v. 2.0. If a copy of the MPL was not distributed with this
# file, You can obtain one at http://mozilla.org/MPL/2.0/.
#
# Covered Software is provided under this License on an "as is"
# basis, without warranty of any kind, either expressed, implied, or
# statutory, including, without limitation, warranties that the
# Covered Software is free of defects, merchantable, fit for a
# particular purpose or non-infringing.
# See the Mozilla Public License v. 2.0 for more details.
#
# For more details, see http://www.mrtrix.org/.

import inspect, os, sys
from collections import namedtuple
try:
  from shlex import quote
except ImportError:
  from pipes import quote
from mrtrix3._version import __version__



class MRtrixBaseError(Exception):
  pass

class MRtrixError(MRtrixBaseError): #pylint: disable=unused-variable
  pass

...