Ask a vague question, get a vague answer
Analysis of variance is a dangerous tool. It allows researchers to avoid asking precise questions. And if you don’t formulate your question precisely, then the answer won’t tell you anything. It’s like the gag in the Hitch Hiker’s Guide where they build a supercomputer to figure out “the answer to life, the universe and everything”. When the computer finally figures it out, the answer is 42. The problem, you see, was that they hadn’t actually figured out what the question was.
Let’s have a look at a simple case. Here is data on cardiac output late in pregnancy in three groups of mothers: those with normal blood pressure, pre-eclampsia (PET) and gestational hypertension (GH)
. table outcome, c(mean co5) format(%2.1f)
----------------------
Hypertens |
ion |
outcome | mean(co5)
----------+-----------
Normal | 6.4
P E T | 5.6
Gest HT | 9.0
----------------------
I used the format option to limit us to one place of decimals.
What does an anova tell us
. anova co5 outcome
Number of obs = 256 R-squared = 0.4984
Root MSE = .790256 Adj R-squared = 0.4944
Source | Partial SS df MS F Prob > F
-----------+----------------------------------------------------
Model | 156.97329 2 78.4866448 125.68 0.0000
|
outcome | 156.97329 2 78.4866448 125.68 0.0000
|
Residual | 157.999502 253 .624503962
-----------+----------------------------------------------------
Total | 314.972792 255 1.23518742
Now, if you find that edifying you’re a better person than I. What the anova tells us is that there is a difference in cardiac output between the three groups. But that’s not the answer to any particular question. It certainly doesn’t answer any useful clinical question.
“Oh, but now we can do post-hoc comparisons”, you say. To which I might reply “But don’t you have any ideas you want to test?” The trouble with post-hoc comparisons is that you can rapidly end up with a bunch of comparisons, some of which are of interest and some meaningless.
First ask a question
To analyse data, you need to articulate the underlying hypothesis. And here, the hypothesis wasn’t “there’s some kinda difference in cardiac output between the three groups”. This is what I call the Empty Brain hypothesis, that assumes no previous research has been done, we know nothing about physiology, about hæmodynamics, about anything. And it’s not good enough. Science progresses by building on our understanding, not by wandering around hypothesising “some kinda difference”.
In fact, we expect that late in pregnancy, cardiac output in gestational hypertension will be higher than normal, causing high blood pressure because it exceeds the normal carrying capacity of the mother’s circulatory system. On the other hand, we expect that it will be below normal in pre-eclampsia because pre-eclampsia is characterised by a lot of clinical indicators of inadequate blood supply. The high blood pressure in pre-eclampsia is the result of very high peripheral resistance (the circulatory system basically closed tight) and the heart desperately trying to get the blood supply through.
Then use regression to answer it
We can use regression to ask this question:
. regress co5 i.outcome
Source | SS df MS Number of obs = 256
-------------+------------------------------ F( 2, 253) = 125.68
Model | 156.97329 2 78.4866448 Prob > F = 0.0000
Residual | 157.999502 253 .624503962 R-squared = 0.4984
-------------+------------------------------ Adj R-squared = 0.4944
Total | 314.972792 255 1.23518742 Root MSE = .79026
------------------------------------------------------------------------------
co5 | Coef. Std. Err. t P>|t| [95% Conf. Interval]
-------------+----------------------------------------------------------------
outcome |
P E T | -.8146901 .217851 -3.74 0.000 -1.243722 -.3856577
Gest HT | 2.612142 .1732165 15.08 0.000 2.271012 2.953272
|
_cons | 6.376119 .0534005 119.40 0.000 6.270953 6.481285
------------------------------------------------------------------------------
I have used Stata’s ‘i’ notation to tell Stata that outcome is to be treated as separate categories. In this case, Stata will use the normotensive group (coded as zero) as the reference group.
Not just hypothesis tests – effect sizes too
The regression has tested our two hypotheses:
1. Compared with women with normal BP, those with PET will have lower cardiac outputs
2. Compared with women with normal BP, those with GH will have higher cardiac outputs
Furthermore, it has quantified the effects. Cardiac output is 0·8 litres a minute lower in pre-eclampsia, and 2·6 litres a minute higher in gestational hypertension.
Robust variance estimation as well!
Another advantage of regression is that we can use Stata’s robust variance estimates. This topic is so important that it will be the subject of its own blog post. But note what happens when I invoke robust variance estimation:
. regress co5 i.outcome, robust
Linear regression Number of obs = 256
F( 2, 253) = 207.37
Prob > F = 0.0000
R-squared = 0.4984
Root MSE = .79026
------------------------------------------------------------------------------
| Robust
co5 | Coef. Std. Err. t P>|t| [95% Conf. Interval]
-------------+----------------------------------------------------------------
outcome |
P E T | -.8146901 .1947096 -4.18 0.000 -1.198148 -.4312319
Gest HT | 2.612142 .1352127 19.32 0.000 2.345856 2.878428
|
_cons | 6.376119 .0549773 115.98 0.000 6.267847 6.48439
------------------------------------------------------------------------------
Our estimates for the effect sizes haven’t changed, but the confidence intervals have. Robust variance estimation allows us to take into account the clustering within the data due to factors beyond our control. Always a good idea!
No comments:
Post a Comment