Dwipreproc error in HPC cluster: permission denied

Dear MRtrix Experts

I am doing preprocessing of my DWI data with a college HPC.
Each subject’s data is submitted as a subjob by PBS.
So with each subject, it works under a ${TMPDIR} folder which is assigned by the cluster.

I can perform the mrconvert, dwidenoise and mrdegibbs smoothly.
However, within the dwipreproc command, it encounter an error when it tries to call dwi2mask.

The following is the error log:

dwipreproc: e[03;32mGenerated temporary directory: /tmp/pbs.2109798[8].cx1/sub-008/DWI_preproc/dwipreproc-tmp-TPXI7V/e[0m
e[03;36mCommand:e[0m mrconvert /var/tmp/pbs.2109798[8].cx1/sub-008/DWI_preproc/sub-008_dwi_denoised_unringed.mif /tmp/pbs.2109798[8].cx1/sub-008/DWI_preproc/dwipreproc-tmp-TPXI7V/dwi.mif
dwipreproc: e[03;32mChanging to temporary directory (/tmp/pbs.2109798[8].cx1/sub-008/DWI_preproc/dwipreproc-tmp-TPXI7V/)e[0m
dwipreproc: e[00;31m[WARNING] Total readout time not provided at command-line; assuming sane default of 0.1e[0m
e[03;36mCommand:e[0m mrinfo dwi.mif -export_grad_mrtrix grad.b
e[03;36mCommand:e[0m dwi2mask dwi.mif - | maskfilter - dilate - | mrconvert - mask.nii -datatype float32 -stride -1,+2,+3
dwipreproc: e[03;32me[0m
dwipreproc: e[01;31m[ERROR] Command failed: dwi2mask dwi.mif - | maskfilter - dilate - | mrconvert - mask.nii -datatype float32 -stride -1,+2,+3e[0me[03;34m (dwipreproc:508)e[0m
dwipreproc: e[03;32mOutput of failed command:e[0m
dwi2mask: computing dwi brain mask… dwi2mask: finding min/max of “mean b=0 image”… [==================================================]
dwi2mask: optimising threshold… done
dwi2mask: thresholding… [==================================================]
.dwi2mask: finding min/max of “mean b=1000 image”… [==================================================]
dwi2mask: optimising threshold… . done
dwi2mask: thresholding… [==================================================]
. done
dwi2mask: [ERROR] error creating temporary file in current working directory: Permission denied
dwi2mask: [ERROR] error creating image “-”
maskfilter: [ERROR] no filename supplied to standard input (broken pipe?)
maskfilter: [ERROR] error opening image “-”
mrconvert: [ERROR] no filename supplied to standard input (broken pipe?)
mrconvert: [ERROR] error opening image “-”
dwipreproc: e[03;32mChanging back to original directory (/tmp/pbs.2109798[8].cx1/sub-008/DWI_preproc)e[0m
dwipreproc: e[00;31mScript failed while executing the command: dwi2mask dwi.mif - | maskfilter - dilate - | mrconvert - mask.nii -datatype float32 -stride -1,+2,+3e[0m
dwipreproc: e[00;31mFor debugging, inspect contents of temporary directory: /tmp/pbs.2109798[8].cx1/sub-008/DWI_preproc/dwipreproc-tmp-TPXI7V/e[0m

The temporary folder has been created by dwipreproc and every file down to the grad.b has been written into the temporary folder.
However, when the “dwi2mask dwi.mif - | maskfilter - dilate - | mrconvert - mask.nii -datatype float32 -stride -1,+2,+3” command is called, it generates error mainly stating:“error creating temporary file in current working directory: Permission denied.”

I am not sure what exactly is happening here, since the temporary folder has been created and the temporary files before dwi2mask command has been created correctly.
Is there a possibility that the temporary folder created by dwipreproc is confused with ${TMPDIR}?

Has anyone encounter similar situation? Or know how to solve the problem?

Thank you very much!

Best wishes,
Chen-Chia

I think the issue might be that under the PBS subjob system, I only have the permission to write in /tmp/pbs.2109798[8].cx1 but not the /tmp itself. However, this is a default behavior of MRtrix for temporal images in pipeline call.

How can I change the default value, I can’t seem to find a mrtrix.config in this kind of HPC environment.
Any pointers?

Thank you very much!

You can set TmpFileDir (and ScriptTmpDir) in the config file or if these are not defined, you can set the environment variable MRTRIX_TMPFILE_DIR as part of the submitted job: https://github.com/MRtrix3/mrtrix3/pull/1407

Thank you!
I define the user config file and it worked.