I would not expect any of the currently-available
5ttgen algorithms to work out-of-the-box for mouse brain data, as they have some component that is implicitly dependent upon human brain data (e.g.
5ttgen fsl involves registration to a brain template in
standard_space_roi and the sub-cortical grey matter segmentation by FIRST is entirely driven by human brain data).
The current set of algorithms is however not intended to be exhaustive, and I have always been eager for users to implement algorithms that will take the segmentation outcomes from other approaches and arrange them into the 5TT format. Indeed the
5ttgen gif algorithm was contributed from outside of the MRtrix3 team; the code shows how it’s a relatively simple conversion from the output that is generated by an external software tool into the 5TT format. So the same could be done for any other segmentation software. Indeed if you make a duplicate of one of the
.py files in
lib/mrtrix3/_5ttgen/ and modify this line to reflect the name of the new file,
5ttgen will detect it and offer access to that new algorithm at the command-line interface. So this is a fairly accessible way of making a contribution to MRtrix3 that could be useful to others.
As far as the actual software to use for mouse brain segmentation, that’s beyond my own expertise. You may find that some certain tissue segmentation tools are agnostic to brain shape (e.g. just running FSL’s
fast may well work).