Dwipreproc error in HPC cluster: permission denied


#1

Dear MRtrix Experts

I am doing preprocessing of my DWI data with a college HPC.
Each subject’s data is submitted as a subjob by PBS.
So with each subject, it works under a ${TMPDIR} folder which is assigned by the cluster.

I can perform the mrconvert, dwidenoise and mrdegibbs smoothly.
However, within the dwipreproc command, it encounter an error when it tries to call dwi2mask.

The following is the error log:

dwipreproc: e[03;32mGenerated temporary directory: /tmp/pbs.2109798[8].cx1/sub-008/DWI_preproc/dwipreproc-tmp-TPXI7V/e[0m
e[03;36mCommand:e[0m mrconvert /var/tmp/pbs.2109798[8].cx1/sub-008/DWI_preproc/sub-008_dwi_denoised_unringed.mif /tmp/pbs.2109798[8].cx1/sub-008/DWI_preproc/dwipreproc-tmp-TPXI7V/dwi.mif
dwipreproc: e[03;32mChanging to temporary directory (/tmp/pbs.2109798[8].cx1/sub-008/DWI_preproc/dwipreproc-tmp-TPXI7V/)e[0m
dwipreproc: e[00;31m[WARNING] Total readout time not provided at command-line; assuming sane default of 0.1e[0m
e[03;36mCommand:e[0m mrinfo dwi.mif -export_grad_mrtrix grad.b
e[03;36mCommand:e[0m dwi2mask dwi.mif - | maskfilter - dilate - | mrconvert - mask.nii -datatype float32 -stride -1,+2,+3
dwipreproc: e[03;32me[0m
dwipreproc: e[01;31m[ERROR] Command failed: dwi2mask dwi.mif - | maskfilter - dilate - | mrconvert - mask.nii -datatype float32 -stride -1,+2,+3e[0me[03;34m (dwipreproc:508)e[0m
dwipreproc: e[03;32mOutput of failed command:e[0m
dwi2mask: computing dwi brain mask… dwi2mask: finding min/max of “mean b=0 image”… [==================================================]
dwi2mask: optimising threshold… done
dwi2mask: thresholding… [==================================================]
.dwi2mask: finding min/max of “mean b=1000 image”… [==================================================]
dwi2mask: optimising threshold… . done
dwi2mask: thresholding… [==================================================]
. done
dwi2mask: [ERROR] error creating temporary file in current working directory: Permission denied
dwi2mask: [ERROR] error creating image “-”
maskfilter: [ERROR] no filename supplied to standard input (broken pipe?)
maskfilter: [ERROR] error opening image “-”
mrconvert: [ERROR] no filename supplied to standard input (broken pipe?)
mrconvert: [ERROR] error opening image “-”
dwipreproc: e[03;32mChanging back to original directory (/tmp/pbs.2109798[8].cx1/sub-008/DWI_preproc)e[0m
dwipreproc: e[00;31mScript failed while executing the command: dwi2mask dwi.mif - | maskfilter - dilate - | mrconvert - mask.nii -datatype float32 -stride -1,+2,+3e[0m
dwipreproc: e[00;31mFor debugging, inspect contents of temporary directory: /tmp/pbs.2109798[8].cx1/sub-008/DWI_preproc/dwipreproc-tmp-TPXI7V/e[0m

The temporary folder has been created by dwipreproc and every file down to the grad.b has been written into the temporary folder.
However, when the “dwi2mask dwi.mif - | maskfilter - dilate - | mrconvert - mask.nii -datatype float32 -stride -1,+2,+3” command is called, it generates error mainly stating:“error creating temporary file in current working directory: Permission denied.”

I am not sure what exactly is happening here, since the temporary folder has been created and the temporary files before dwi2mask command has been created correctly.
Is there a possibility that the temporary folder created by dwipreproc is confused with ${TMPDIR}?

Has anyone encounter similar situation? Or know how to solve the problem?

Thank you very much!

Best wishes,
Chen-Chia


#2

I think the issue might be that under the PBS subjob system, I only have the permission to write in /tmp/pbs.2109798[8].cx1 but not the /tmp itself. However, this is a default behavior of MRtrix for temporal images in pipeline call.

How can I change the default value, I can’t seem to find a mrtrix.config in this kind of HPC environment.
Any pointers?

Thank you very much!


#3

You can set TmpFileDir (and ScriptTmpDir) in the config file or if these are not defined, you can set the environment variable MRTRIX_TMPFILE_DIR as part of the submitted job: https://github.com/MRtrix3/mrtrix3/pull/1407


#4

Thank you!
I define the user config file and it worked.