Scalable surveillance of e-cigarette products on Instagram and TikTok using computer vision.

Researchers

Journal

Modalities

Models

Abstract

Instagram and TikTok, video-based social media platforms popular among adolescents, contain tobacco-related content despite the platforms’ policies prohibiting substance-related posts. Prior research identified themes in e-cigarette-related social media posts using qualitative or text-based machine learning methods. We developed an image-based computer vision model to identify e-cigarette products in social media images and videos.We created a dataset of 6,999 Instagram images labeled for 8 object classes: mod or pod devices, e-juice containers, packaging boxes, nicotine warning labels, e-juice flavors, e-cigarette brand names, and smoke clouds. We trained a DyHead object detection model using a Swin-Large backbone, evaluated the model’s performance on 20 Instagram and TikTok videos, and applied the model to 14,072 e-cigarette-related promotional TikTok videos (2019-2022; 10,276,485 frames).The model achieved the following mean average precision scores on the image test set: e-juice container: 0.89; pod device: 0.67; mod device: 0.54; packaging box: 0.84; nicotine warning label: 0.86; e-cigarette brand name: 0.71; e-juice flavor name: 0.89; and smoke cloud: 0.46. The largest number of TikTok videos – 9,091 (65%) – contained smoke clouds, followed by mod and pod devices detected in 6,667 (47%) and 5,949 (42%) videos respectively. Prevalence of nicotine warning labels was the lowest, detected in 980 videos (7%).Deep learning-based object detection technology enables automated analysis of visual posts on social media. Our computer vision model can detect the presence of e-cigarettes products in images and videos, providing valuable surveillance data for tobacco regulatory science.Prior research identified themes in e-cigarette-related social media posts using qualitative or text-based machine learning methods. We developed an image-based computer vision model to identify e-cigarette products in social media images and videos.We trained a DyHead object detection model using a Swin-Large backbone, evaluated the model’s performance on 20 Instagram and TikTok videos featuring at least two e-cigarette objects, and applied the model to 14,072 e-cigarette-related promotional TikTok videos (2019-2022; 10,276,485 frames).The deep learning model can be used for automated, scalable surveillance of image- and video-based e-cigarette-related promotional content on social media, providing valuable data for tobacco regulatory science. Social media platforms could use computer vision to identify tobacco-related imagery and remove it promptly, which could reduce adolescents’ exposure to tobacco content online.© The Author(s) 2023. Published by Oxford University Press on behalf of the Society for Research on Nicotine and Tobacco.