Hi @olie,
I’ve split off your original reply as a new topic, as it was a bit hidden in an existing topic (that was about another topic
). I hope you don’t mind. 
I’ll get back to your questions tomorrow; I’ve got some good answers, but it may take some time to write them up decently… I’ve also got a figure or two to help illustrate some things for you.
In the mean time, a few quick remarks on the commands you’ve posted; as I’m spotting an error in there and I’ve got another piece of advise. It may be best to test (again) using these points, so as to make sure we’re still talking about the same findings:
-
The
-lmax 8,0,0indwi2response(bothdhollanderandmsmt_5tt) is wrong: the-lmaxoption fordwi2responsespecifies lmax’es for each b-value rather than for each tissue type. For these algorithms, the lmax’es that you specify only apply to the WM response, as the GM and CSF ones are inherently always isotropic (lmax=0) for all b-values. So in your case, this should have been something like-lmax 0,8,8instead, reflecting the WM lmax for b=0, b=700 and b=3000. However, it’s even better to not supply the lmax option at all! The lmax is now set automatically to 10 for you for all non-b=0 shells of the WM response. Don’t worry about your 60 directions not being enough for that or something: the response is estimated from at least hundreds of voxels, so lmax=10 (and in principle even way beyond) is easily supported for the response functions. So long story short: ditch the lmax option todwi2response.
-
I spotted a suspicious
-mask mask_dilated.mifoption todwi2responsethere. The suspicious thing (for me) being the “dilated” there. This may make sense todwi2fod, so you’re covering a safe extra margin when computing FODs, but fordwi2responsethis isn’t needed. In fact, this may even be detrimental there: as no useful response functions will ever be obtained from that margin beyond the brain and outer CSF. In extra-fact,dwi2response dhollandereven on purpose erodes the brain mask: the definition/intuition of a “safety margin” for the purpose of response functions works quite the other way around: you’ve got loads of voxels in a whole brain mask, so you can be extra-picky about choosing only the best ones. It makes sense to avoid dangerous areas (like the very outer few voxels of a potentially not-entirely-accurate mask) at any cost. Your data looks like relatively normal developed brains, in which casedwi2responsewill happily compute a decent (enough) brain mask itself, and erode it for you. So long story short for this one: ditch the-maskoption todwi2responseas well.
-
Your
dwi2fodcall is fine (and correct), but you can safely ditch the-lmaxoption here too. It’ll manage the lmax’es safely on its own for you.
So I would recommend re-running both dwi2response calls (and then dwi2fod of course as well) without the -mask and -lmax options, and let them do their thing that way. 
Maybe a small additional request, so I can help you better (tomorrow when I work on a reply for you): could you zoom into a region (somewhere, anywhere around the cortex). The current screen shots are very hard to look at for those regions, as the resolution (of the screen shot) is quite low. It’s impossible to see what those FODs look like around these areas. 
Finally, definitely also provide a copy-paste of the text (numbers) in each response function file (they’re just text files), e.g. as was done in this post. They typically explain much more about your response functions than the FODs (indirectly) do. The interactions in multi-tissue CSD can get quite complicated between the 3 tissue types at times, depending on certain properties of the actual response functions.
Ok, that was already quite some text. I hope my actual reply tomorrow won’t be much longer… 
Cheers,
Thijs