Jenn’s Journal Club: Quantifying Scientific Impact

So since my current job involves a lot of reading of various journals, I come across a wide range of interesting papers on a regular basis. I thought I’d start sharing my thoughts occasionally if one really jumped out at me and today I think I have a perfect inaugural subject for this series: the study of scientific impact [1]. Impact in scientific publishing is a fascinating topic, to scientists and publishers alike. But for me, questions of impact speak to a deeper sense of self as a scientist. We constantly ask ourselves how we are doing professionally. We don’t necessarily get raises, promotions, or new titles often, but our work is constantly moving and changing. We can look at how well each paper does, but how can we tell if we’re poised for greatness, past our prime, or just plain missed the boat?

There is a feeling in Western society, at least, that people who are going to be great tend to show greatness from a young age. We look up to hear stories about Mozart, Pascal, and Gauss showing remarkable talent at a young age and we feel inadequate by the time we’re in high school. We hear about Bill Gates becoming a technology pioneer after dropping out of college and we feel inadequate by the time we graduate college and haven’t had our big break. And yet, the truth is that most scientists don’t have that one big break. This paper looked at the ones who did have a big break: Nobel Laureates.

For years, it’s been accepted that any most-significant output of a creative or scientific mind will come earlier in the career rather than later. And if you create a plot of the probability of a highly-significant (i.e., cited) paper from a scientist versus the time since that scientist’s first publication, you will find that this holds true. However, this plot is skewed by two things: one is that most scientists have a dropoff in publication frequency after an initial early-career rush, and the second is that papers published early have a longer time to gain citations. In order to remove these from the calculation, the authors plotted a different probability.

The authors of this study looked at over 200,000 authors who had been publishing in Physical Review journals for at least 20 years. They looked at a random paper in a given sequence of papers and plotted the probability that a paper will be significant versus its relative position in a history. So basically, how likely a paper is to be significant based on where it is in line. And with this change, they found that the probability distribution flattened, suggesting that the probability of a significant paper over time is actually random. They go on to fully develop and support this random-impact model, as they call it.

And this is major. This means that no matter when in your career you publish a paper, there is a chance it could be your most significant paper, over time. Sure, the paper that’s been out for 20 years is more likely to have more citations than the paper that’s been out for five years, but over time, this will flatten. Significant papers are random events, like cosmic ray bursts of genius. Which means that we don’t need to spend our youth frantically trying to make our mark and our middle years bemoaning that it never came.

1. Barabasi, et al, Science354, aaf5239 (2016).

Advertisements
Tagged , ,
%d bloggers like this: