# Dwipreproc command error; memory issue?

**URL:** <https://community.mrtrix.org/t/dwipreproc-command-error-memory-issue/425>\
**Category:** Uncategorized\
**Tags:** error, performance\
**Created:** [September 6, 2016, 4:45am UTC](https://community.mrtrix.org/t/dwipreproc-command-error-memory-issue/425 "2016-09-06T04:45:43Z")\
**Posts on this page:** 1\
**Showing post:** 2

<div class="post-metadata">

**Author:** ![rsmith](https://community.mrtrix.org/user_avatar/community.mrtrix.org/rsmith/32/2672_2.png) [@rsmith](https://community.mrtrix.org/u/rsmith)\
**Post date:** [September 6, 2016, 5:30am UTC](https://community.mrtrix.org/t/dwipreproc-command-error-memory-issue/425/2 "2016-09-06T05:30:57Z")

</div>

Yes, I would definitely suspect a memory issue in this instance, as previously experienced [here](https://community.mrtrix.org/t/error-of-running-population-template/395) and [here](https://community.mrtrix.org/t/error-response-function/393). The clue is in the output provided after this line:

```auto
dwipreproc: Output of failed command:

```

It subsequently gives details of `mrinfo` not receiving the piped image it expects, but it’s `mrconvert` that is genuinely failing, and that command is _not providing any terminal output at all_.

> **Click here for the technical details of what's going on if you're interested**
>
> The issue is that it’s a very low-level system operation that goes awry. The memory-mapped output file gets truncated by the system without the memory-mapping code being made aware of it. When the code subsequently tries to access beyond the actual memory-mapped file size, it generates a bus error signal, which calls `terminate()`.
> 
> When running a binary command directly, this will generate an error at the command-line such as: “`Bus Error (core dumped)`”; which is useful but maybe not verbose enough. However by default `glibc` writes this message to the _current tty terminal_ rather than `stderr`. When executed within one of the Python scripts using `subprocess`, this message is lost.
> 
> Theoretically, setting the environment variable `LIBC_FATAL_STDERR_` is supposed to instruct the signal handler to instead write the error to `stderr`; but in my experience this didn’t work.
> 
> Therefore, the actual solution is to register our own signal handlers that will be called in the event of a fatal system signal. Apart from customising the error messages, and making sure that they are written to `stderr` such that they can be piped appropriately, we can also perform limited cleanup in the event of catastrophic command failure: for instance, deleting any temporary image files used in piping.
> 
> * * *

Once we have confirmed and merged in the code changes [here](https://github.com/MRtrix3/mrtrix3/pull/776), both the _MRtrix3_ binaries and scripts should give more meaningful error messages when such a system error is encountered. Note however that this will require re-running the `configure` script to activate the changes, and is not guaranteed to be supported on all systems.

We are also considering modifying the Python scripts to create the temporary working directory in the _current working directory_ by default, rather than using `/tmp/` by default, in the hope that such memory issues are encountered less frequently. It will still be possible to set the temporary directory location manually using the `-tempdir` option if you know you have enough space in e.g. `/tmp`, and we’ll probably add a config file entry to control this behaviour as well.

Cheers  
Rob

---

_[View the full topic](https://community.mrtrix.org/t/dwipreproc-command-error-memory-issue/425)._