Knock Knock Eye: How Step One Pass/Fail Fueled the Research Arms Race

So last episode I made the case that medical student research should be banned, and I clearly touched a nerve. I got flooded with angry messages arguing against me, and honestly, I expected some backlash since it was a genuinely spicy take. But I want to be clear, I was being facetious, obviously nobody's going to actually ban med student research. What I actually want to unpack this episode is why this topic clearly struck such a nerve, and what's really driving the explosion in med student publications.

I walk through some new data, including a study that found roughly a quarter of recent ophthalmology journal articles may contain AI-generated text, and another from Yale showing a real gap between the publications applicants claim on their residency applications and what's actually verifiable in PubMed. I also get into Goodhart's Law, the idea that when a measure becomes a target, it stops being a good measure, and how that applies perfectly to what's happened since Step One went pass-fail. I share some thoughts on what might actually help, including capping how many publications students can list, and I make it clear I'm not trying to lead some crusade here, I just think this conversation needed to happen.

Takeaways:

Since Step One became pass-fail, medical students have shifted their focus to research output as a differentiator, and reported publication numbers have risen sharply as a result.

A recent study found that roughly 26% of ophthalmology journal abstracts and 22% of commentaries screened in 2025 scored as likely containing AI-generated text.

A Yale-affiliated study found that applicants to ophthalmology residency claimed an average of 4.9 publications, while only 2.4 were actually verifiable in PubMed.

Goodhart's Law suggests that once a measure like publication count becomes a target used for residency selection, it stops accurately reflecting research quality or applicant ability.

Starting with the 2027 application cycle, the AAMC is renaming the ERAS publications field to "scholarly works" in an attempt to shift emphasis from quantity to quality.

To Get Tickets to Wife & Death: You can visit Glaucomflecken.com/live 

We want to hear YOUR stories (and medical puns)! Shoot us an email and say hi! knockknockhi@human-content.com

Can’t get enough of us? Shucks. You can support the show on Patreon for early episode access, exclusive bonus shows, livestream hangouts, and much more! –⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠ http://www.patreon.com/glaucomflecken⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠ 

Also, be sure to check out the newsletter: ⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠https://glaucomflecken.com/glauc-to-me/⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠