While driving to work this week, I was nearly driven off the road by a man who was speeding, smoking a cigarette with one hand, and texting with the other. I swerved into the next lane over, and once the norepinephrine was cleared from my circulation, my thought process led me to think about a phosphatonin that has made a tremendous splash in the past few years.
Hold your judgment; I think about weird things in the car.
What made this guy such a bad driver? What was he doing wrong? Could I have known he was a bad driver before he tried to prematurely end my days? These questions actually illustrate something that is not explained well in the world of medical papers: the difference between explanatory and predictive models.
In brief, explanatory models attempt to determine cause, while predictive models attempt to determine risk. Let's use what happened to me to explain this.
Imagine that I'm interested in studying two major factors that may influence whether you'll drive a nice, young biostatistician off the road: speeding and texting (there are clearly others). After some Googling, I determined that the breakdown of what people were doing prior to accidents is:
We're missing two critical elements here. 1) I don't know how these behaviors overlap. 2) I don't know how often people engage in these behaviors when they are NOT getting in accidents. If I were to draw a Venn diagram, it could look like this:
Venn Diagram A: 40% of accidents have drivers that were texting, while 10% of accidents have drivers that were speeding, but none were doing both. Any relationship between these behaviors and an accident is purely coincidental.
It could also look like this:
Venn Diagram B: 40% of accidents have drivers who were texting, and 10% have drivers who were speeding, but now there is overlap between drivers who are speeding and texting.
In the above simulation, speeding causes accidents much more than texting does. Does your brain hurt yet? Mine does. If anything, this example illustrates why Venn diagrams are limited to middle school homework. At the very least, it is clear that the question of what causes an accident is a hard one to address.
How best to use this information?
But if, like me, you want to protect the burgeoning population of commuting numbersmiths, which population should we target? Which behavior should we try to change?
Surprisingly, those questions are not the same. If you want to target a "high-risk" population, you are asking a predictive question, and need to build a predictive model. However, if you want to determine what behavior to change, you need an explanatory model.
Let's first decide who to target by building a predictive model. Should we focus on people who speed, or people who text? For these examples, I'll use the assumptions from Venn diagram B. Speeding increases the risk of accident more than texting, but speeders are also more likely to text.
We should target the speeders, right? I literally just told you that speeding increases the risk of accident more than texting. But texting increases the risk too -- and there are many more of those people. Plus, if we target texters, we'll get a bunch of speeders included since those two behaviors are correlated.
We need to find a factor that is not only prevalent in people who cause accidents, but actually distinguishes people who cause accidents from those who don't. This is a classic problem in predictive statistics, and is usually addressed with an "area under the receiver-operator curve (ROC)" or "c-statistic" -- both examples of discriminant functions (because they measure the ability of a factor to discriminate between who will and will not have an outcome of interest). I mention these technical terms only so that when you see them in an article, you may say: "A-ha! A predictive model. I shall proceed to share this with my spouse and/or co-workers."
In the framework of Venn diagram B, identifying people who text is more discriminant than identifying people who speed. So, we target texters.
But do we target texting? Should we lock the phones? Should we auto-correct "driving home" to "I'm a d-bag"?
To answer this, we need an explanatory model.
As our assumptions from Venn diagram B tell us, a proportion of texters might be speeders too, and there is a higher risk of an accident from speeding compared with texting. To untie these factors, the statistical technique employed is called "multivariable adjustment" and will certainly be its own Methods Man topic. Multivariable adjustment attempts to strip away all these confusing intercorrelations and get at the root cause of why these accidents happen. Our explanatory model clearly shows that speeding is a bigger problem than texting.
We can actually reconcile these results by considering the question we are trying to answer. If the governor knocks on your door and says, "I have a driver re-education program I want to try out. What group of people is at highest risk for an accident?" you'd use your predictive model and say "Texters!" If instead the governor says, "Should I spend 1 million dollars on a program to stop texting or a program to stop speeding?" you'd break out your explanatory model and say "Speeding!"
Thankfully, often the explanatory and predictive models show you that the same factor is the most discriminant (predictive) and most strongly associated (explanatory). But not always.
So, what does this have to do with phosphatonins?
A recent JASN article by Rebholz et al. entitled "Serum Fibroblast Growth Factor-23 Is Associated with Incident Kidney Disease" gives us a great medical literature example of how these very different statistical approaches can get confused at times.
FGF-23 was discovered in 2000, but it was a decade later when its role as the major regulator of phosphate in the kidney was appreciated. It is an important protein (upregulation gives you autosomal dominant hypophosphatemic rickets, downregulation gives you familial tumor calcinosis). And due to its tight relationship with metabolism in the kidney, it is biologically plausible that it might have something to do with the development of chronic kidney disease (CKD).
With that in mind, the CKD-biomarkers consortium got together and used an existing cohort (in this case the ARIC cohort) to ask a simple question: If we measure the baseline level of FGF-23, can we predict who will go on to develop kidney disease?
The authors found a statistically significant relationship between the baseline FGF-23 level and the development of CKD and ESKD (P<0.001). Participants in the highest quintile of FGF-23 developed ESKD at a rate nearly five times as fast as those in the lowest quintile. This persisted after adjusting for all types of relevant factors like baseline kidney function (your "explanatory model" sense should now be tingling).
But can we use FGF-23 to predict? Let's look at the numbers. The authors tell us that the rate of ESKD is five times as high in the highest quintile of FGF-23 (levels greater than 54.6 pg/mL) compared with the lowest quintile (levels less than 32 pg/mL).
There are essentially three groups in this study. High FGF people, low FGF people, and those stuck in the middle. Let's put the raw numbers into these groups.
So if you have a high value, what's your chance of getting ESKD? Of the 2,689 people with a high value, 113 went on to have an event (4%). In other words, if we set off the alarms when an FGF-23 is >54.6 pg/mL, we'd be unnecessarily worried about 96% of people. The positive predictive value (obviously, a predictive statistic) is 4%.
The strong association in the explanatory model tells us that maybe FGF-23 could be a therapeutic target. That's exciting. But, as far as a predictor goes, it is not a home run.
And that's a fact
And this is the critical difference between explanatory statistics and predictive statistics. They both have their uses. In short, a predictive study tells you who to enroll in your trial of a novel drug. An explanatory study tells you what the drug should be. FGF-23 is a fascinating molecule with biology that is being nicely explained through both basic and epidemiologic science. But I predict its use as a biomarker to be limited.
Stay safe out there.
The Methods Man is F. Perry Wilson, Assistant Professor of Medicine at the Yale School of Medicine. He earned his BA from Harvard University, graduating with honors with a degree in biochemistry. He then attended Columbia College of Physicians and Surgeons in New York City. From there he moved to Philadelphia to complete his internal medicine residency and nephrology fellowship at the Hospital of the University of Pennsylvania. During his research time, Wilson also obtained a Masters of Science in Clinical Epidemiology from the University of Pennsylvania. He is an accomplished author of many scientific articles and holds several NIH grants. If you'd like to see more of his work, please visit him at www.methodsman.com or follow @methodsmanmd on Twitter.