Understanding GLM global intercept

2

Hi Ana,

Splitting this off simply because the title of the other thread doesn’t really match the content anymore (which makes it harder for others to find).

Can you explain when the added column of 1s would be explicitly necessary?

I’m not sure to where in the original thread you are referring, I don’t think I explicitly said that it is necessary; but nevertheless, depending on the rest of your design matrix it may be necessary, or indeed it may be unwanted.

Imagine that in your design matrix you encode sex in a single column as such:

-1 -3
 1 -2
 1  1
-1  3
 1  1

Now let’s say that you are testing FD, and that the value observed for males is 1.1, and the value observed for females is 0.9. The fit of the design matrix above to these data will be terrible. It simply can’t represent the input values using the axes of freedom that it has available. In fact I did the experiment (with vectorstats -notest; handy for understanding GLMs). It gives beta coefficients of: [0.3, 0.0]. Now the values predicted by the model are the inner product between these beta coefficients and the values of the explanatory variables (i.e. the variables that go into the design matrix). This means that the predictions of this model are:

  • For a male of mean age ( [1 0] ): FD = (0.3 * 1) + (0.0 * 0) = 0.3
  • For a female of mean age ( [-1 0] ): FD = (0.3 * -1) + (0.0 * 0) = -0.3 :upside_down_face:

But with a column of ones added:

1 -1 -3
1  1 -2
1  1  1
1 -1  3
1  1  1

When this model gets inverted against these data, the beta coefficient are: [1.0, 0.1, 0.0]. The predicted values are now:

  • For a male of mean age ( [1 1 0] ): FD = (1.0 * 1) + (0.1 * 1) + (0.0 * 0) = 1.1
  • For a female of mean age ( [1 -1 0]): FD = (1.0 * 1) + (0.1 * -1) + (0.0 * 0) = 0.9

Now contrast this against your design matrix, where instead of having a single column classifying group, you have two columns, each encoding membership of one group:

0  1 -3
1  0 -2
1  0  1
0  1  3
1  0  1

Now the beta coefficients are: [1.1, 0.9, 0.0], and the predictions are:

  • For a male of mean age ( [1 0 0] ): FD = (1.1 * 1) + (0.9 * 0) + (0.0 * 0) = 1.1
  • For a female of mean age ( [0 1 0]): FD = (1.1 * 0) + (0.9 * 1) + (0.0 * 0) = 0.9
    So either of the last two designs works, as long as each is interpreted correctly.

But here’s why the idea of a column of ones being “explicitly necessary” is too general. Consider adding a GI to your design matrix:

1  0  1 -3
1  1  0 -2
1  1  0  1
1  0  1  3
1  1  0  1

What will the beta coefficients be? Well, they could be [0.0 1.1 0.9 0.0], such that the second and third columns still encode the sex means. Or it could be [1000.0 -999.9 -1000.1 0.0]; that fits the provided data just as well. This is rank-deficiency.

What you said about “role of the ones column is to estimate the “global intercept”, which is the value of the dependent variable when all explanatory variables are zero.” confused me.

This is me once again being hyper-precise, sacrificing a potential lack of understanding for potential mis-understandings.

Consider this case again:

1 -1 -3
1  1 -2
1  1  1
1 -1  3
1  1  1

, and remember that the value of the exploratory variable (the input; FD in this case) that is predicted by the model is the inner product between the beta coefficients and the explanatory variables.

Now consider performing this inner product with the vector [1 0 0] ("all explanatory variables are zero"). The result of this inner product will be to simply extract the value of the first beta coefficient; the one attributed to the column of ones; the “global intercept”. What this represents is the predicted value of FD for a hypothetical individual with “a value of sex of 0” (interpret as you will) and a value of age of 0 (assuming you have demeaned your age data, this would be the mean age of the cohort). In this case, the value of this beta coefficient would be 1.0, which is the average of the mean male FD and the mean female FD, and therefore seems a reasonable predictor of the value of FD for any hypothetical participant about which no information is known.

As such:

Are you saying that FD/FDC/FC would be 1 when all EVs are zero?

No, the other way around: when all EVs are zero, the global intercept is the value of FD / FDC / FC.

Also, are the numwaves/numpoints/matrix lines necessary?

These are entirely FSL GLM constructs, with which I have zero familiarity. MRtrix3 statistical inference commands expect each requisite feature of the model (e.g. design, contrast) to be provided as its own text file containing numerical matrix data.

Cheers
Rob

2 Likes