Weighing the Odds

Robert DeRubeis had a problem. For years, as the Samuel H. Preston Term Professor in the Social Sciences and Chair of Psychology researched depression, he saw study after study that looked at just one variable of the treatment—for example, whether cognitive therapy or medication was better for patients who also had a personality disorder.
 
With real people, of course, there’s never just one variable. What if he had data showing that one treatment was better for men, but the other was better for patients over 60, and he was treating a 65-year-old man? Clinicians must examine the separate studies, consider what they know about the patient, and make a decision. DeRubeis thought there should be a better way. “It bugged me,” he says.

He talked to colleagues across Penn, and discussed it each year with his graduate students and post-doctoral fellows. Now, they’ve developed the “personalized advantage index,” which can simultaneously consider and assign a weight and value to each variable that may affect treatment and make a recommendation on the best way to go.

DeRubeis and his team created the index using data from a study like the example above, of medication versus cognitive therapy. They looked at the results and did follow-up research for all 154 participants in the study. They then used a “leave one out” strategy, in which they predicted the results for each patient using the data from the other 153, to develop an algorithm that would determine how much each factor should be considered and the preferred treatment for each patient. The result: If the PAI had been used to assign the study’s patients to their ideal treatment options, the difference in success would have been equivalent to the difference between medication and a placebo in a blind study.

“There’s so much being written about personalized medicine today,” says Zachary Cohen, a doctoral student in psychology and second author of the study. “As we move toward greater use of genetic tests and neuroimaging, integrating all of those things into treatment recommendations is going to become increasingly difficult. [The personalized advantage index gives] the clinician a way to actually look a patient in the face and say, here’s what these factors mean for you. You, a unique individual.”

The index also lets researchers and clinicians combine data from decades’ worth of studies in a useful way. DeRubeis says, “Our aim was to create interest in a very promising approach to the use of data that are just sitting around doing nothing.”

They’ve gotten the interest. Since the paper was published in the online journal PLOS ONE, they’ve heard from groups around the world. Their next step will use work they’ve done with Wharton statistics doctoral students Adam Kapelner and Justin Bleich. They’ve adapted the Bayesian Additive Regression Trees (BART) technique developed by Wharton Universal Furniture Professor of Statistics Ed George to take the personalized advantage index into machine learning, allowing them to compare variables in complex, non-linear ways. “It’s not just there for our little lab to use,” says Cohen. “Anybody who wants to implement treatment selection around the world can download this software.”

The tool isn’t limited to psychology, or even, eventually, to medicine. “We want this thing to go viral,” says DeRubeis. “Not the particulars so much as the recognition that data can be used in a vastly more productive way to help with delivering the best treatments.”