Critical analysis of common methodology flaws in e-cigarette surveys

The prevalence of vaping, also known as using e-cigarettes, vapes, vape pens, or electronic nicotine delivery systems (ENDS) has prompted a demand for reliable, evidence-based research.¹ However, published literature on the topic of vaping is often unreliable, characterized by serious flaws and a failure to adhere to accepted scientific methodologies. In this narrative review, we analyze 24 popular vaping studies, published in medical journals, that purport to evaluate the association of vaping and smoking initiation, smoking cessation or health outcomes. We analyzed these studies to identify the questions they claimed to address, stated methods, manner of implementation, discussions, and stated conclusions. After critical appraisal, we noted a multiplicity of flaws in these studies, and identified patterns as to the nature of such flaws. Many studies lacked a clear hypothesis statement: to the extent that a hypothesis could be inferred, the methods were not tailored to address the question of interest. Moreover, main outcome measures were poorly identified, and data analysis was further complicated by failure to control for confounding factors. The body of literature on “gateway” theory the for initiation of smoking was particularly unreliable. Overall, the results and discussion contained numerous unreliable assertions due to poor methods, including data collection that lacked relevance, and assertions that were unfounded. Many researchers claimed to find a causal association while not supporting such findings with meaningful data: the discussions and conclusions of such studies were therefore misleading. Herein, we identify the common flaws in the study design, methodology, and implementation found in published vaping studies. Our aim is to prompt future researchers to adhere to scientific methods to produce more reliable findings and conclusions in the field of vaping research.

Critical analysis of common methodology flaws in e-cigarette surveys

Carl V Phillips¹, Cother Hajat², Emma Stein³, Riccardo Polosa^4,5, and the CoEHAR study group⁶

1. Independent Researcher
2. Public Health Institute, UAE University
3. Independent Researcher
4. Center of Excellence for the Acceleration of HArm Reduction (CoEHAR), University of Catania, Catania, Italy.
5. Department of Clinical and Experimental Medicine, University of Catania, Catania, Italy.
6. Center of Excellence for the Acceleration of Harm Reduction, University of Catania, Italy. Full list of author information is available at the acknowledgment section.