Connectomestats NBS Output Interpretation

Hello,
I am using connectome sstats to run NBS and NBSE on diffusion MRI data.
I am having some trouble interpreting the results and kind find help in the documentation, unfortunately.

For NBS I am running:
connectomestats nbs_files.txt nbse designMatrix.txt contrast.txt nbs_output

And for NBSE:
connectomestats -threshold 1.5 nbs_files.txt nbs designMatrix.txt contrast.txt nbs_

The result files I get are:
tvalue.csv, std_effect.csv, stv_dev.csv, enhanced.csv, bet1.csv, beta0.csv, abs_effect.csv, uncorrected_pval.csv, null_dist.txt, fwe_pvalue.csv

For NBS I was expecting to obtain one (the biggest) component, that is smaller in size than the input connectivity matrices, together with a p value for that component.
I was expecting something similar for NBSE.

I am assuming that “enhanced.csv” is the component, is that correct?
Can someone help to interpret what each of the files represents?
Thank you!

Hi Lena,

Understanding the output of connectomestats using the nbs algorithm requires a slightly different interpretation of what NBS is doing fundamentally. A large part of the justification of this is that the code path for NBS is almost precisely equivalent to that of NBSE; and indeed is really not all that different to CFE. By making code and algorithms as generalised as possible, not only does it reduce our total volume of code, and mean that improvements to the code can trivially be made available across multiple commands, but it can also provide insight into the fundamental nature of the data manipulations.

Note that NBSE intrinsically performs statistical inference at the edge level. Even though the results it produces most frequently produce connected sub-networks, inference is not performed upon those sub-networks as a whole; each edge is independently statistically significant. The connectedness of these networks is an emergent property of the statistical inference step, resulting from the fact that connect edges enhance the statistical belief in one another from the hypothesis of a common genuine biological effect, but the inference is per-edge. This is similar to CFE, where macroscopic white matter bundles emerge as being statistically significant, even though inference was performed at the fixel level.

As you said (I think), a perhaps more intuitive output from NBS would be: For each connected sub-network that survives p<0.05 following permutation testing (note that there may be more than one), have a text file containing a binary matrix that highlights those edges that are part of the significant sub-network, and possibly also provide the p-value for that network. This would however clearly be quite inconsistent with both NBSE and CFE, which report significance at the edge / fixel level, and would therefore require a considerably different code path.

The above equates to the question:
“Are there any sub-networks that are larger than predicted by chance? If so, show me the network and the p-value.”

connectomestats using the nbs algorithm instead re-formulates this question as:
“Is the size of the sub-network comprising this particular edge larger than predicted by chance?”

Pragmatically this has a number of implications. In both CFE and NBSE, the file enhanced.* provides the enhanced t-statistic value for that fixel / edge, as resulting from statistical enhancement. This then allows the data from each fixel / edge to be compared against the null distribution independently. In NBS, identification of the sub-network size acts as the equivalent of “statistical enhancement”: the original t-statistic is “enhanced” by selecting the size of the sub-network of which all t-values are above some ascribed threshold. Therefore, it is the total size of the connected sub-network that is compared to the null distribution. But instead of doing this once for each sub-network, in connectomestats this value is simply duplicated across all edges comprising the sub-network (these are the values you see in enhanced.csv), and comparison to the null distribution is done (somewhat redundantly) independently for each edge. Hence, despite inference being applied by connectomestats at the edge level rather than the sub-network level (and therefore being entirely in common with NBSE), in the case of NBS the resulting corrected p-value is equivalent for all edges within the sub-network, and hence the entire sub-network is invariably extracted as a whole when applying a p-value threshold.

Hopefully this makes sense…

Regarding the other output files, we probably need to add somewhere (either in the online documentation or in the commands’ help pages) descriptions of the output files, since they’re consistent across all statistical inference commands.

Rob

Hi Rob,
thanks a lot for the useful response. This helps me understand connectomestats much better! One (hopefully) last question: I am using NBSE now and am looking at the fwe_pvalue file.
I am trying to extract the subnetwork from it. If I got you right, I will simply need to threshold this matrix to obtain the subnetwork. I am noticing however that there are several different p-values in this file, which I take to mean that there are several subnetworks. Just to confirm: A different p-value in this file means the connection belonging to a different subnetwork, is that correct? And the p-value that I see here is the significance of the respective subnetwork as a whole?
Thanks a lot for you help!

I am trying to extract the subnetwork from it. If I got you right, I will simply need to threshold this matrix to obtain the subnetwork. I am noticing however that there are several different p-values in this file, which I take to mean that there are several subnetworks.

I think this might be a fundamental misunderstanding about how the NBSE method works. Hopefully the manuscript is not too far away now, which will explain it all in much more detail. But I’ll try to address this as succinctly as possible.

In NBS, sub-networks are extracted based on a cluster-forming threshold, and the size (or mass) of each sub-network is compared to the null distribution to produce a p-value for each sub-network. So for each sub-network, statistical inference is applied to that sub-network as a whole.

In NBSE, an enhanced t-statistic is produced for each edge, and these values are compared to the null distribution to produce a p-value for each edge. So while applying a particular p-value threshold may result in the appearance of a sub-network, this is an emergent effect only; statistical inference is still applied to each edge individually.

1 Like