Home » Blog » Brooke De Lench » Research Papers and Peer-Reviewed Studies: A World of Difference

 

Last week, we posted to the site a group of four articles about a peer-reviewed study in the Journal of Neurosurgery showing that football helmet design affected concussion risk among a large group (or what scientists call a "cohort") of college football players.

Because the study's findings matched almost exactly the estimated difference in concussion risk in the STAR helmet ratings, a test developed by the co-lead authors of the study, Virginia Tech-Wake Forest's Stefan Duma and Steven Rowson, which is designed to measure the effectiveness of football helmets in reducing the forces that cause concussion, we also updated an existing article about those ratings to include a reference to the new study. Young football player drinking from squeeze bottle

Just before those articles were published, we received a press release from the American Academy of Neurology about an abstract of a research paper on football helmets slated for presentation to the AAN Annual Meeting in Philadelphia in late April.  The press release said that, using crash test dummies, researchers in Florida had found that football helmets do "very little" to protect kids against the rotational forces that cause concussion, and quoted the co-lead author, Dr. Frank Conidi, director of the Florida Center for Headache and Sports Neurology and Assistant Clinical Professor of Neurology at Florida State University College of Medicine, as saying that the study dispelled the assumption of "generations of football and other sports participants" that their brains are protected by their investment" in football helmets.

While numerous newspapers and reporters - with at least one notable exception (see below) - elected to take the research paper largely at face value in publishing stories (links to which then popped up repeatedly in our Twitter feed and were dutifully retweeted), MomsTEAM took a pass, which led many to wonder, on Twitter and via email, why?

For a number of reasons.

First, as many of you know, MomsTEAM has been covering the concussion beat for 14 years, longer than anyone in the mainstream media.  Since I and my staff, especially Senior Health and Safety Editor Lindsay Barton, began wriiting about concusssions in general, and fooball helmets in particular, back in 2000, we have tried to live up to our reputation as the "trusted source" for youth sports safety information by making sure that anything we publish has been properly vetted and is based on methodologically sound scientific research.

As a result, we have become skeptics, critics and curators of credible information on the subject.  In a now very crowded space of concussion activists, bloggers, and a largely uneducated media, we continue to see our mission of keeping all sports active kids safe by separating fact and fiction, and making sure what we publish, while it may not be the first to discuss a particular development, is the most comprehensive and objective.

Second, all we had to report on was an abstract of a research paper.  With so many researchers vying for limited funding dollars and attention, the trend these days seems to be a game of jumping the gun, sending out press releases about abstracts of papers before they have appeared, if they ever do, in peer-reviewed journal papers.  While MomsTEAM occasionally reports on research papers, we are careful to point out that they have not been subject to scrutiny by the scientific community. 

In this particular instance, when Lindsay and I dug deeper, our investigation raised in our minds serious concerns that reporting on this particular research paper at this preliminary stage might give it more credence than perhaps - emphasis on the word, "perhaps" - it deserved. Without even talking to anyone with expertise in the subject, we were concerned about reporting on the paper knowing absolutely nothing about the methodology the researchers employed, especially when it involved an unspecified modification to the NOCSAE drop tests, purportedly to capture data about rotational forces.

Capturing data on rotational forces, much less setting helmet standards or designing helmets that reduce such forces, has been a monumental challenge for biomechanical engineers and helmet manufacturers for decades. We have been critical of NOCSAE in the past for its failure to test football helmets for their ability to reduce rotational acceleration (although, as we also recently reported, the organization says it will be announcing a new helmet standard at its next meeting in June which will for the first time include a standard on rotational acceleration). Our original and updated article about the Virginia Tech STAR ratings also includes criticism by NOCSAE that it did not capture data on rotational acceleration (talk about the pot calling the kettle black!). As a result, we needed to be extra cautious in publicizing a paper employing a new methodology.

Third, that the research paper being publicized by AAN was commissioned by BRAINS, Inc., a Florida research and development company employing one of the co-authors, Dr. John Lloyd, also raised a red flag. As we noted in our article about the new Virginia Tech-Wake Forest helmet study - which compared concussion data on an older model of Riddell helmet (the VSR4) versus a newer Riddell model (the Revolution) -  a similar conflict of interest led many in the concussion research community to discount a 2006 study reporting a difference in concussion risk between the Riddell Revolution and other helmets, peer-review notwithstanding, because it was funded by, you guessed it, Riddell.

Fourth, when we did ask respected researchers in the field to comment on the Condini-Lloyd paper, while they declined to comment on the record for MomsTEAM, we can say that they were not at all impressed by what little could be gleaned from the press release, one characterizing it, off the record, as "a train wreck."

And lastly, fifth, we weren't at all clear what the take-home message of the paper is. 

If the point of the paper is to say that current helmets don't protect very well against concussion, and, because they don't, the emphasis needs to be on coaches teaching proper tackling technique, players strengthening their neck and shoulder muscles, and officials more strictly enforcing the rules against helmet-to-helmet contact, then there was nothing new to report to MomsTEAM readers, as the fact that helmets are designed to protect against concussions in the first place, and proper tackling, neck strengthening, and rules enforcement are ways to reduce concussion risk that we have been recommending for years, and are part of Pillar Two of the Six Pillar approach to concussion risk management we outlined in our PBS documentary, "The Smartest Team."

On the other hand, if the point of the paper was to emphasize the point that football helmets should be tested on how well they attenuate rotational forces, which are the principal cause of concussions, and that new testing standards are needed, there was, once again, nothing new on which to report. There is already a well-formed consensus about the need for new standards, a subject on which MomsTEAM and I have reported for years.  (Indeed, as fate would have it, I am on my way to Atlanta today to talk about this very subject at the inaugural meeting of the National Council on Youth Sports Safety's PASS (Protecting Athletes and Sports Safety) initiative as Chair of the Technology & Manufacturing Group, with will be charged with the responsibility, among others, of making recommendations regarding helmet standards.

The bottom line is that, after considering all these factors, MomsTEAM decided not to run a stand-alone article, but to direct our readers instead to our existing, constantly updated, content on football helmets and concussion risk reduction, including these articles:

Study Showing Helmet Design Can Reduce Concussion Risk Leaves Many Questions Unanswered

Virginia Tech Football Helmet Ratings: Helpful But Come With Limitations

Proper Tackling Reduces Risk of Catastrophic Injury Says Ex-Pro

Neck Strengthening: Reducing Concussion Risk In Football

Seven Ways To Reduce Risk of Traumatic Brain Injury In Sports

The Six Pillars of Concussion Risk Management: The MomsTEAM Approach

Making Sure Football Helmet Fits: A Simple, But Effective Way To Minimize Concussion Risk?

Still looking for an article about the Condini-Lloyd study that got it about right? Check out this article by Melissa Healy, one of our favorite sports health and safety reporters, which does a terrific job of at least presenting the major issues the research paper raises, albeit not in the depth that MomsTEAM does. 


Brooke de Lench is Executive Director of the non-profit MomsTEAM Institute, Founder and Publisher of MomsTEAM.com, and Producer/Director/Creator of the PBS documentary, "The Smartest Team: Making High School Football Safer."  You can reach her via email (delench@momsteam.com) and follow her on Twitter @brookedelench.