I could of course (as I currently do) merge all these
.tck files before computing SIFT2 weights, but that requires twice as much disk space.
It’s only disk space Plus if you don’t need to use the original track files individually, you can just delete them upon performing the concatenation.
Please correct me if I am wrong, but it would also be incorrect to compute SIFT2 weights on each individual
.tck files independently,
It’s not strictly “incorrect”, just not ideal. By doing so you are assuming that the density of streamlines in each subset is adequately dense to both reasonably sample all possible reconstruction trajectories, and to provide enough granularity with which SIFT2 can correct the gross reconstruction density biases individually in each; you’re then simply concatenating multiple results together in order to increase total streamline count. The fit is just about guaranteed to be inferior to using all track data at once though. I also can’t rule out the possibility of some systematic bias (or lack of correction thereof) that could become additive across subsets, and hence the quantity of the correction could become inferior to if the entire tractogram were processed as a whole in unusual ways, but I don’t really have a sense of the likely nature or possible magnitude of such effects.
After reading through the GitHub issue, I understand that piping streamlines would solve several problems in one go – it would certainly solve the one I raised. And I appreciate that providing this functionality for one tool would soon lead to extend others too.
It’s worth clarifying that my intention for streamlines data piping was not to use memory-mapping of any sort, but to literally pass the vertex data in binary from from
stdout of one process to
stdin of another, hence obviating the need to store the intermediate data on disk. An intrinsic limitation of such then is that random access is not slow but impossible. I’m certainly not against alternatives, that’s just what I have in mind right now.
I am always impressed with how fast one gets an answer from you or Rob on this forum, kudos really!
Thanks! It is a labour of love…
Speak for yourself; @alan-connelly chained me to my desk years ago and nobody is responding to my cries for help…
The closest to this is probably SIFT/SIFT2, but the streamlines get internally processed as they’re being read to a different representation more suitable for processing – so there’s no need to access the raw streamlines data after that anyway (although it does two passes over the data if I recall correctly)
No, should only be the one; except for
tcksift producing the output tractogram, but it could theoretically as an alternative write a vector of ones and zeroes that
tckedit could then use to retain / remove streamlines.