Statistical Sleuthing: Can you spot a problem?

Can you spot problems in a scientific study just by reading the published paper? We put our statistical sleuthing skills to the test with a famous study on procrastination and deadlines that was retracted more than 20 years after it was published. But were there clues hidden in plain sight in the published results—and would we have noticed anything suspicious at the time? We put on our data-detective hats and find a few statistical clues of our own before turning to Data Colada’s investigation of the original data, where the story gets even stranger. Along the way, we hunt for missing participants, question suspiciously enormous effects, meet some unlikely data twins, and discover why sometimes the best statistical tool is simply asking: Do these data behave like real people?

Statistical topics

Statistical sleuthingEffect sizesDegrees of freedomReplicationMissing dataMethodologic Morals

“A data detective's toolbox is surprisingly simple. Check the degrees of freedom, look for twins, question giant effects, and ask whether the data actually make human sense.”“You don't have to be a statistician to catch a statistical criminal.”

References

Data Colada blog posts:https://datacolada.org/138https://datacolada.org/139Data and code: https://researchbox.org/7135Retraction Watch article: https://retractionwatch.com/2026/09/03/procrastination-study-duke-dan-ariely-psychological-science-data-colada-tampering-retraction/ Original 2002 paper: https://journals.sagepub.com/doi/10.1111/1467-9280.00441 Retraction notice: https://journals.sagepub.com/doi/10.1177/09567976261488042 Hyndman K, Bisin A. Replication of "Procrastination, Deadlines, and Performance: Self-Control by Precommitment". Psychol Sci. 2026;37(8):557-571. doi:10.1177/09567976261460772Kristin and Regina’s online courses:

Demystifying Data: A Modern Approach to Statistical Understanding

Clinical Trials: Design, Strategy, and Analysis

Medical Statistics Certificate Program

Writing in the Sciences

Epidemiology and Clinical Research Graduate Certificate Program

Program that Kristin teaches in::