I need some advice regarding the best way to warp an ROI from MNI space to a population FOD template generated for a study.
I have been playing around with registering average images but I suspect that that is not the best way to go about it (though I’ve got a reasonable result - other than brainstem structures).
The multiplication of warps (your mrcalc lines) does not yield the composition of the warps. As you found out, the correct way to “concatenate” two warps is using transformcompose.
The warps used in transformcompose are deformation fields (each voxel value points to the location in the source space (x,y,z)). The rim you see are most likely interpolation artefacts (I guess in the warp mni2subject) due to setting the warp outside the edge of the FOV to zero. This is sensible for displacement fields, not for deformation fields where these values now point close to the location (0,0,0).
I think the rim should not appear if the warps are both generated using mrregister. Were all warps generated by mrregister?
If not, what tool did you use to generate the warp (transformed the output of warpinit) and did you apply warpcorrect to these warps? warpcorrect fixes these (0,0,0) deformation field values. You can inspect the edge of the FOV of these individual warp images (jet colourmap helps) to see if there are 0s or close to 0 values in all 3 volumes. You can remove the outermost voxels of problematic warps for instance with mrgrid warp.mif crop -uniform 1 - | mrcalc - 0 -mult 1 -add - | bin/mrgrid - pad -uniform 1 -fill nan - | mrcalc warp.mif - -mult warp_corrected.mif (use mrcrop and mrpad if you are using the current master branch as mrgrid is still only available in the dev branch`).