The National Academies of Science, Engineering and Medicine just published Sponsor Influences on the Quality and Independence of Health Research, which sums up a meeting where experts discussed how scientific studies funded by industry or conducted by an author with a conflict of interest consistently report results more favorable to industry or the sponsor than studies free from bias. The report finds similar patterns across research areas including tobacco, nutrition, and pharmaceutical industries.
The report summarizes how industries can bias or undermine the research process in often hard-to-detect ways by controlling:
- The questions asked
- Details of study design and analysis
- Conduct of the study
- Which results they publish and don’t publish
- Whether industry connections and involvement with the study are fully and accurately disclosed
It is a summary of the information presented at the meeting, not a report by the National Academies that draws conclusions or makes recommendations on behalf of the Academy, but it is nonetheless very informative.
Here are the goals for the meeting that NASEM established:
- the sources of funding of health research, including during the life cycle of knowledge generation, from original research to reanalysis and replication to clinical and public health information dissemination,
- evidence regarding whether source of funding influences study quality
and outcomes, and - models, processes, and principles used to protect the independence
and quality of research.
Here are some of the points in the report:
- For some industries (such as pharma) industry-sponsored studies report statistically significant results and/or larger effect sizes more often than non-industry-sponsored studies; for others (such as tobacco) industry-sponsored studies are more likely to report smaller effects or results that do not reach statistical significance.
- Bias can appear at four points in the research process: when the agenda is
set and how the question is asked, the method and its internal validity, how
the study is conducted, and whether the results are published in full. - Corporate entities that manufacture doubt and uncertainty in science are a
threat to human health. - Open science, which includes increased transparency and access to data,
can help address the risk of explicit and implicit bias. - Industry funding biases areas of research and marketing in the field of nutrition,
which can lead to public health harm. - Publication bias, or selective publication of study findings, can be misleading.
There are studies that were conducted but were not published. Without
publishing all research, we do not have the full picture. - One speaker saw AI technology, such as natural language processing and machine learning, as a way to identify publications with outcome reporting bias. (I have my doubts because AI “learns” from the internet and it would be possible to bias the AI engine by flooding the internet with pro-industry results.)
The report also discusses potential solutions to protect the research and publication process from industry bias and includes some examples from organizations who are trying to minimize bias in the work they fund and do. Unfortunately, the report does not address the key question of how regulatory agencies, policy makers, the media and the public should account for these well-established biases when considering studies industry submits to influence decision making.
Full citation: National Academies of Sciences, Engineering, and Medicine. 2023. Sponsor
Influences on the Quality and Independence of Health Research: Proceedings of a
Workshop. Washington, DC: The National Academies Press. https://doi.org/10.17226/27056. It is available here.
Stanton Glantz blog 