Dear Mrtrix Team,
At the moment I got this error “Segmentation fault: Invalid memory access”, when I do dwidenoising and when I try to load a fixel plot. Both commands have already been working for a week ago.
I tried to start dwi denoise this way: dwidenoise dwi.mif denoised.mif -noise noise.mif
When I start the command a second time another error occurs: dwidenoise dwi.mif denoised.mif -noise noise.mif -force dwidenoise(3221,0x7fffa75003c0) malloc: *** mach_vm_map(size=703687441776640000) failed (error code=3) *** error: can’t allocate region *** set a breakpoint in malloc_error_break to debug libc++abi.dylib: terminating with uncaught exception of type std::bad_alloc: std::bad_alloc Abort trap: 6
Another strange thing is that when I tried to load a fixel map in mrview (via tools load fixel) the same first error occurs:
[ERROR] unknown format for image "…/fixel/directions.nii"
mrview: [ERROR] error opening image “…/fixel/directions.nii”
I already tried to find a solution in the community but I could not fix it.
Thank you for any kind of help. I really appreciate this great support-site!
Best
Max
Strange… Can you provide information about what OS you’re running on? Also, what version of MRtrix3 (dwidenoise -version)? And some information about the input image might be useful - in particular if it’s large: it looks like malloc() is failing to allocate a chunk of ~625 petabytes, which is not surprising, but we’d need to figure out why it tries to do this in the first place…
Also, what is the full output when running with the -debug command? This will give us an idea of which step is failing. Also, can you try running without multi-threading (-nthread 0) to see whether the issue relates to concurrency. And if you have time and the inclination, can you try running this within the debugger:
gdb --args dwidenoise dwi.mif denoised.mif -noise noise.mif -force -debug -nthread 0
...
(gdb) run
...
[ allow program to run & crash ]
...
(gdb) bt
And post the full output of that run.
Also, you mention that this command works OK the first time? So is this an intermittent problem? If so, I would suspect some issue with your system rather than MRtrix per se. One possible issue is that you might have performed a system software update recently, but not rebooted, which might lead to instabilities due to mismatches between the (old) running kernel and the updated system libraries, or issues of that nature.
Not sure about the second problem with mrview, but given your issues with dwidenoise, I strongly suspect fixing one will fix the other…
Thank you for the quick reply!
I am a bit confused, I try the dwidenoise on some other data and it worked well. Just all my data measured with reversed phase encoding direction (blipdown) don’t work.
I started dwidenoise again in debug mode.
So this is the result:
lip-osx-001145:~ wichmann$ cd /Users/.....wichmann$ dwidenoise DTIblipdown224grappa2.mif DTIblipdown224grappa2_denoised.mif -noise noised.mif -force -debug -nthread 0
dwidenoise: [WARNING] existing output files will be overwritten_
dwidenoise: [DEBUG] No config file found at "/etc/mrtrix.conf"_
dwidenoise: [DEBUG] No config file found at ".mrtrix.conf"_
dwidenoise: [INFO] opening image "DTIblipdown224grappa2.mif"..._
dwidenoise: [DEBUG] reading key/value file "DTIblipdown224grappa2.mif"..._
dwidenoise: [DEBUG] sanitising image information..._
dwidenoise: [DEBUG] memory-mapping file "DTIblipdown224grappa2.mif"..._
dwidenoise: [DEBUG] file "DTIblipdown224grappa2.mif" mapped at 0x10ae40000, size 1404928 (read-only)_
dwidenoise: [DEBUG] image "DTIblipdown224grappa2.mif" loaded_
dwidenoise: [DEBUG] image "DTIblipdown224grappa2.mif" initialised with strides = [ -1 -112 12544 ], start = 12543, using indirect IO_
dwidenoise: [100%] preloading data for "DTIblipdown224grappa2.mif"_
dwidenoise: [DEBUG] image "DTIblipdown224grappa2.mif" initialised with strides = [ -1 -112 12544 ], start = 12543, using direct IO_
dwidenoise: [INFO] creating image "DTIblipdown224grappa2_denoised.mif"..._
dwidenoise: [DEBUG] sanitising image information..._
dwidenoise: [DEBUG] creating empty file "DTIblipdown224grappa2_denoised.mif"_
dwidenoise: [DEBUG] resizing file "DTIblipdown224grappa2_denoised.mif" to 2810700_
dwidenoise: [DEBUG] sanitising image information..._
dwidenoise: [DEBUG] memory-mapping file "DTIblipdown224grappa2_denoised.mif"..._
dwidenoise: [DEBUG] "DTIblipdown224grappa2_denoised.mif" appears to reside on a networked filesystem - using delayed write-back_
dwidenoise: [DEBUG] file "DTIblipdown224grappa2_denoised.mif" held in RAM at 0x10b246000, size 2809856_
dwidenoise: [DEBUG] image "DTIblipdown224grappa2_denoised.mif" loaded_
dwidenoise: [DEBUG] image "DTIblipdown224grappa2_denoised.mif" initialised with strides = [ -1 -112 12544 ], start = 12543, using direct IO_
dwidenoise: [INFO] creating image "noised.mif"..._
dwidenoise: [DEBUG] sanitising image information..._
dwidenoise: [DEBUG] creating empty file "noised.mif"_
dwidenoise: [DEBUG] resizing file "noised.mif" to 2810700_
dwidenoise: [DEBUG] sanitising image information..._
dwidenoise: [DEBUG] memory-mapping file "noised.mif"..._
dwidenoise: [DEBUG] "noised.mif" appears to reside on a networked filesystem - using delayed write-back_
dwidenoise: [DEBUG] file "noised.mif" held in RAM at 0x10b4f4000, size 2809856_
dwidenoise: [DEBUG] image "noised.mif" loaded_
dwidenoise: [DEBUG] image "noised.mif" initialised with strides = [ -1 -112 12544 ], start = 12543, using direct IO_
dwidenoise(20909,0x7fffa75003c0) malloc: *** mach_vm_map(size=773940465106944000) failed (error code=3)_
*** error: can't allocate region_
*** set a breakpoint in malloc_error_break to debug_
libc++abi.dylib: terminating with uncaught exception of type std::bad_alloc: std::bad_alloc_
Abort trap: 6_
My current version is:
== dwidenoise 5beeb0c5 ==
64 bit release version, built Jun 22 2017, using Eigen 3.3.4
Ah, OK. I think the clue is here (a bit cryptic, I’ll admit):
Looks like your image is a single 3D volume (only 3 strides listed). This can’t work with dwidenoise, which relies on redundancy in the volume dimension (and/or the spatial neighbourhood). It can only operate on 4D data. Looks like we need to catch this explicitly and provide a more helpful error message…
Thank you for this explanation.
Is it possible to use dwidenoise when I combine all my reversed phase encoding data to one 4d file and start the command again?
Thanks
Max
This is a question that comes up quite a bit. I’m not sure what the right answer is here, but I reckon there’s no harm in trying. Just make sure to inspect your residuals for any undesired structure, as suggested in the docs…
Is it possible to use dwidenoise when I combine all my reversed phase encoding data to one 4d file and start the command again?
Depends on how much “reversed phase encoding data” you have. If it’s literally 2 volumes, the second with a reversed phase-encoding direction, then there may still not be enough image data within the sliding window. Unless you make the sliding window huge, in which case the requirement of constant noise level may no longer apply.
In total I have measured 5 reversed phase encoding data. For each of my 5 sessions (different acquisition schemes) one reversed phase encoding direction.
I will give it a try.
Thank you
Max
So I combined all reversed phase encoding images and started dwi denoising.
I also checked the residuals but I do not know how to interpret them. When I compare them with the residuals from the normal DWI’s they look different. On the suggested docs I read that the less anatomical structure is seen the better the denoising. I guess something went wrong with my denoising because I can see some structure???
Denoising of my combined reversed phase encoding images:
Sorry, coming back to this thread after some time away - hope it’s still useful.
The results on the b2850 DWI series look OK (no clear structure at all - not sure why it’s in colour though?)/. The others on the other hand are a bit suspect…
The b700 results clearly show structure, but that is limited to the ventricles, so might still be OK. It probably reflect an unstable b=0 signal in the CSF (leading to increased residuals), combined with a genuinely redundant signal (no orientation dependence), allowing dwidenoise to estimate the noise level more accurately. The fact that the noise appears so low in the rest of the brain is likely a reflection of the fact that 11 volumes is too small a number to observe any redundancy in the signal, so the algorithm defaults to no or very little denoising - hence a very small noise level estimate. I’m guessing the good news there is that it’s only denoised in regions where it could estimate the noise, and left the bulk of the parenchyma signal broadly untouched - probably not what you were after, but the safer option nonetheless.
The results on your reversed phase encoding images have even more structure. It also looks like the noise map is essentially zero apart from the small number of voxels that do have signal - again indicating no denoising for these voxels. This is really not surprising given how little redundancy there would be with 5 volumes only…
However, coming back to your original statement here:
I realise I’d misunderstood what you were suggesting. What I thought you were talking about was merging your main DWI series with the reversed phase-encoding data to end up with a single dataset with lots of volumes. There is then the possibility of redundancy in that domain, even if the images don’t align properly - but as I’d hinted before, I don’t think anyone has really looked into whether this works as intended or not…
One final note: do you need to denoise all of the data? Assuming your phase-reversed data were acquired purely for the purposes of distortion correction, then you should be able to extract the matching b=0 volumes from the non-denoised main DWI series, and merge this with the non-denoised reversed phase-encoding images using mrcat, and feed that to dwipreproc using the -rpe_pair -se_epi options (more info on these options on the dwipreproc help page). No doubt @rsmith will correct me if I’m getting any of this wrong…
