Glad to hear from long-time MRrtix users! 
So the issue of getting a clean set of responses for multi-shell analysis in your case is made more complicated for two reasons:
-
it’s not human, so doesn’t necessarily match many of the assumptions built into some of the algorithms, but most notably the dhollander approach, which relies on heuristics tuned specifically for in vivo adult human data (@ThijsDhollander might be able to comment further here).
-
it’s ex vivo, which changes the contrast properties quite a bit, and again will completely invalidate the assumptions implicit in the dhollander method.
It’s generally accepted that when scanning ex vivo, you need b-values ~4× what you would use in vivo for roughly equivalent contrast. So your b = 7200 s/mm² is equivalent to just under b = 2000 s/mm² in vivo, which isn’t that high. So I don’t understand why the tensor model would fail this badly, either using the full set or just the single-shell variant. Any chance you could post a snapshot of the FA map? It might give use a clue as to what might be going wrong…
Otherwise, to get decent multi-shell responses, you will need to resort to a more manual approach. The simplest is probably to use the manual algorithm, feeding in masks containing voxels that you are confident are representative of ‘pure’ tissue of interest. If you use MRView’s ROI editor and select a few voxels of pure CSF and GM, and save these as CSF_voxels.mif and GM_voxels.mif respectively, you can then estimate those responses with:
dwi2response manual dwi.mif CSF_voxels.mif CSF_response.txt -lmax 0,0,0,0
and similarly for the GM. Note this assumes you’ve concatenated all of your shell data together into one big dataset using mrcat -axis 3.
The WM is a little bit more tricky in that you want to select voxels that contain pure white matter, but also that contain only a single fibre orientation. The simplest is probably to use the same trick as in the old version (and employed in the dwi2response fa algorithm): erode a brain mask, and select the N highest FA voxels within it (note this relies on having a good FA map to begin with…):
maskfilter brain_mask.mif erode -npass 5 - | mrcalc fa.mif - -mult - | mrthreshold - -top 100 WM_voxels.mif
Here I’m eroding the brain mask by 5 voxels, and selecting the top 100 highest FA voxels - you may need to adjust that to fit your data. After that, you can again use the manual algorithm to get the WM response:
dwi2response manual dwi.mif WM_voxels.mif WM_response.txt -lmax 0,4,6,8
In this case, we’re asking for a lmax = 0 for the b = 0 shell, then lmax = 4 for shell 1, lmax = 6 for shell 2, and lmax = 8 for the last shell. Again, these are numbers you can probably afford to play with - higher is probably something we’d consider safer these days, given recent improvements in the response estimation (as provided by amp2response).
Hopefully that’ll allow you to analyse your data - sounds like quite an impressive acquisition. But it would be good to get to the bottom of the issue you’re having with the highest shell…