I’m running a modified version of the pipeline for fixel-based analysis (single-shell data). The only thing being changed is the directory structures.
When it comes to the fixelcorrespondence and warp2metric options (sections 17 and 18), I’m running into the following errors for a subset of patients (around 8 out of ~35):
fixelcorrespondence: [ERROR] file “/working/lab_michaebr/alistaiP/Park/AFD3/templateoutput_test/fixels/fd/index.mif” is smaller than expected
warp2metric: [ERROR] file “/working/lab_michaebr/alistaiP/Park/AFD3/templateoutput_test/fixels/fc/index.mif” is smaller than expected
An example of the command where this issue occurs, is:
The directories are structured in that the original index.mif from fod2fixel (section 13) is stored in the directory fixel_mask. The fd file for example, of Patient_039, is stored in a parent folder called fd under Patient_039.mif. The subject-specific folders that are generated in section 15 of the documentation are also stored in a parent directory called fixels/intemplatespace.
I note that this index file (within the fd folder) is updated as fixelcorrespondence progresses, as per the command history of the file.
Are there any tips to troubleshoot this particular error? I’ve looked at the debug so far but nothing has alerted me to any issues.
I can provide access to my github repos (that calls these scripts) to help troubleshoot.
That’s a slightly concerning one: the error isn’t actually specific to the new fixel format in any way, but is a more general warning from the image reading step. Based on the offset of the start of the image data from the start of the file, the image dimensions, and the datatype, it’s possible to predict what size the output image should be; if it doesn’t match, this error is thrown.
My best guess is that you’re running fixelcorrespondence & warp2metric in multiple parallel jobs, and possibly also on a network-based file system?
One of the “quirks” of these commands I disagreed with during development is that the index and directions files of the output fixel directories are always over-written, regardless of whether or not they already exist. The rationale behind this was that the command invocation would be exactly the same, regardless of whether or not the output fixel directory already existed, thereby avoiding an additional explicit step in the pipeline of making a copy of the fixel template beforehand. It seems that in your case, two jobs are simultaneously attempting to resize / testing the size of the same file.
So I suspect that simply running these particular jobs sequentially across subjects should bypass the problem. Definitely let us know if this is in fact not the case.
setting foreach -1 has done the trick. I can still keep the same amount of cpu usage by inputting -nthreads 10 to each instance of the command.
I should have mentioned that these errors led to further warnings with "error converting string" in mrcalc (either in section 18 or 19). Such errors are non-existent now.
I would definitely suggest this peculiarity to be added to the documentation
I guess this is something we should do something about…? Maybe switch to verify rather than overwrite if the index file & directions files already exist - which would actually seem to be a good idea to catch potential mismatches in the directions file. Not guaranteed to work correctly the first time though, if the command that first creates the files updates them slower than they get verified by another concurrent command. Not sure how best to handle this - worth creating a GitHub issue in any case?
