Introduction

Cervical cancer is the fourth most common cancer among women worldwide (1). Epidemiologically and molecularly, the human papillomavirus (HPV) is a circular, double-stranded DNA virus known to cause chronic infections and is the primary cause of cervical cancer (2). Other cancers associated with HPV include vulvar, vaginal, anal, rectal, penile, and oropharyngeal cancers (3).

Among HPV types, 16 and 18 are high oncogenic risk strains responsible for over 70% of cervical cancer cases. The Gardasil-9 vaccine covers seven of the 15 high-oncogenic risk strains, including HPV types 16 and 18. The vaccine has been shown to be effective against these strains and has been considered safe based on studies and long-term follow-ups (4). Systematic reviews have shown that the HPV vaccine is highly effective in reducing the incidence of HPV infections and related cancers, as well as anogenital warts in all genders. Accordingly, the World Health Organization recommends a one- or two-dose schedule for individuals aged 9–14 years, primarily targeting girls, but applicable to all individuals within this age group (5). For those who miss this age range, it is recommended to receive the vaccine before the first sexual experience or up to the age of 26. For individuals aged 27–45, vaccination decisions should be made jointly by the physician and the patient. Vaccination ages may vary by country (6). The HPV vaccine is part of the national vaccination programs in many countries due to its high cost-effectiveness and safety; however, it is not included in Türkiye’s national vaccination program (6,7).

A study conducted in Türkiye in 2022 showed that although 67.3% of women had heard of the HPV virus, only 3.6% had received the HPV vaccine. The prevalence of HPV positivity in this population was also high at 14.2% (8).

Social media platforms enable both content creators and health-care providers (HCPs) to disseminate health-related information to a broad audience. Examining the influence of social media on health literacy is crucial, as digital platforms have become primary sources of health information. Nearly one-third of Americans obtain news online, positioning social media as a critical channel for reaching diverse communities, particularly those with limited health literacy. Smartphone-dependent populations, including low-income and minority groups, rely heavily on social media for health information, highlighting the need for accessible, clear, and culturally appropriate health communication (9). Young people increasingly prefer digital methods to access information about HPV and HPV vaccination, seeking accurate content through various online platforms (10). However, health-related content on these platforms is often shared without a medical accuracy assessment. YouTube, the most widely used platform for health topics, offers both advantages and disadvantages (11). While it facilitates easy access to information, it also enables the rapid dissemination of misinformation, which can undermine herd immunity and pose public health risks (9).

According to the most recent Digital 2024 Global Overview, the majority of YouTube users are aged 25-34 years, constituting 21.3% of the user base. The second most frequent age group is 35-44 years, followed by young adults aged 18–24 years, who account for 15.5% of YouTube’s user base (12). This demographic alignment with the routine vaccination group (ages 9–13), the catch-up vaccination group (ages 13–26), or the joint decision-making group (ages 27–45), as well as parents or caregivers of potential vaccine recipients. This alignment underscores the importance of examining HPV vaccine-related content on YouTube.

Social media is a potential resource for improving health literacy in Türkiye (13). However, using YouTube without awareness can pose potential risks. HCPs may not always be able to keep up with the rapidly increasing social media content, but it is clear that these contents significantly affect public health and patients’ quality of life. This study aimed to evaluate the fluency, understandability, actionability, quality, and accuracy of Turkish-language video content related to the HPV vaccine on YouTube, emphasizing the importance of responsible and cautious content sharing on social media.

Materials and Methods

This study is a descriptive, cross-sectional content analysis of publicly available Turkish-language YouTube videos related to HPV vaccination.

Data Collection and Management

Because the study was based on retrospective analysis of publicly accessible online content, ethics committee approval was not required. 

Video searches were conducted on YouTube in May 2024 using the following Turkish-language keywords: HPV aşısı (HPV vaccine), Gardasil aşısı (Gardasil vaccine), and serviks kanseri aşısı (cervical cancer vaccine). Searches were performed in incognito mode and without logging into Google or YouTube accounts to minimize algorithm-driven personalization and promote unbiased results. Analysis of the search terms revealed that all videos retrieved by the keyword Gardasil were also present in the results for Gardasil aşısı, whereas the reverse was not true. Therefore, Gardasil aşısı was selected as the final keyword to maximize inclusion of relevant videos. The first 50 videos for each keyword were screened and included. 

Figure 1. The flowchart of the video selection.

Videos shorter than 15 seconds, longer than 15 minutes, silent videos, videos in languages other than Turkish, and duplicate videos were excluded. Figure 1 shows the flowchart of the video selection method.

The source of each video was categorized according to the type of YouTube channel, and the speaker’s profession was identified when available. This information was obtained from publicly accessible descriptions, profile sections, or visual or verbal content presented on the respective YouTube ­channels.

The top five comments for each video were recorded and evaluated. A descriptive analysis based on data-driven observations was conducted to evaluate the content of these comments.

In the initial phase of the evaluation, all four researchers jointly reviewed a randomly selected sample of 10 videos and scored them according to the predetermined evaluation scales. This collaborative assessment aimed to ensure consistency and standardization in applying the scoring criteria. The remaining videos were then independently evaluated by three researchers. Scores were compared, and in cases of inconsistency, a consensus was reached through consultation with the senior researcher.

Evaluation Scales
Patient Education Material Assessment Tool for Audiovisual Material (PEMAT-A/V)

The Patient Education Material Assessment Tool for Audiovisual Materials (PEMAT-A/V) is a validated tool used to evaluate the understandability and actionability of videos. It comprises 13 items (items 1, 3–5, 8–14, and 18–19) for measuring understandability, and four items (items 20–22 and 25) for measuring actionability. 

Each item is scored as “1” (Agree), “0” (Disagree), or “NA” (Not Applicable). Scores are calculated according to the official scoring manual and expressed as percentages, with higher percentages reflecting greater understandability and actionability (14).

The Journal of the American Medical Association (JAMA) Score

The Journal of the American Medical Association (JAMA) criteria is an internationally recognized scale for assessing the transparency and reliability of medical information on websites. It consists of four steps: 1) Authors and contributors, 2) Attribution (references, copyright information), 3) Transparency (sponsorship, advertising, conflict of interest), and 4) Currency. 

Each criterion is scored as “1” if present and “0” if absent, yielding a total score ranging from 0 to 4. A score of ≥3 indicates high reliability, whereas a score of ≤2 indicates low reliability (15).

Video Power Index (VPI)

The Video Power Index (VPI) is an internationally recognized measure used to assess a video’s popularity. It is calculated by multiplying the total number of likes by the number of views, and then dividing the result by 100. Higher VPI values indicate greater popularity (16).

Video Information and Quality Index (VIQI)

The Video Information and Quality Index (VIQI) is a composite tool used to assess the informational and production quality of health-related videos. It includes four domains: Flow of Information, which measures how well the video presents information in a structured and logical manner; Information Accuracy, which evaluates the correctness and reliability of the content; Visual Quality, which assesses the richness of the video by awarding points for elements such as images, animations, interviews, captions, and summaries (scored on a 4-point scale); and Precision, which determines how well the video title aligns with its actual content. All domains except Visual Quality are rated on a 5-point Likert scale. The total VIQI score ranges from 4 to 19, with higher scores indicating better overall quality (17). Higher VIQI and VPI scores indicate higher quality and popularity, respectively.

Global Quality Score (GQS)

The Global Quality Score (GQS), developed by Bernard et al. (18), is a widely used tool for assessing the overall quality, flow, and patient usefulness of internet-based health information sources. Videos are scored on a 5-point Likert scale: 1 = poor quality, poor flow, most information missing, not helpful for patients; 2 = generally poor, some information provided but of limited usefulness; 3 = moderate quality, with some important information adequately discussed; 4 = good quality and flow, most relevant information covered, useful for patients; 5 = excellent quality and flow, very useful for patients. 

Statistical Analysis

All statistical analyses were performed using SPSS Statistics for Windows, version 29.0 (IBM Corp., Armonk, NY, USA). The normality of the data distribution was assessed using both statistical tests and visual methods. Descriptive data are presented as numbers and percentages for categorical variables, and as median, interquartile range (IQR), and minimum–maximum values for continuous variables. Comparisons of continuous variables between groups were performed using the Mann-Whitney U test. A Type I error rate of 5% was accepted for all analyses.

Results

The study included the top 50 YouTube videos for each of the following keywords: HPV aşısı (HPV vaccine), Gardasil aşısı (Gardasil vaccine), and Serviks kanseri aşısı (Cervical cancer vaccine). After applying the exclusion criteria, the final dataset comprised 81 videos. The median video duration was 95 seconds (IQR, 105 seconds). 

Table 1. Distribution of video duration and evaluation scores (YouTube, May 2024).

Table 2. Distribution of YouTube video channel sources and speakers’ professions (May 2024)

The JAMA scores had a median of 2 (IQR, 1) out of 4, indicating low transparency and reliability. The GQS scores had a median of 3 (IQR, 2) out of 5, reflecting moderate quality. The PEMAT understandability scores had a median of 66% (IQR, 25). Similarly, the PEMAT actionability scores had a median of 66% (IQR, 33), indicating moderate levels of both understandability and actionability. Table 1 shows the distribution of video durations and PEMAT, JAMA, VIQI, VPI, and GQS scores. The distribution of the videos’ channel sources and speakers’ professions is shown in Table 2. 

Table 3. Distribution of video duration, PEMAT, JAMA, VIQI, and GQS scores according to VPI score.

The videos were further divided into two groups based on the VPI score of 144, as the median VPI score (≤144 and >144). The Mann-Whitney U test compared the two groups based on duration, JAMA, VIQI, GQS, and PEMAT, and found no statistically significant differences (p>0.05) (Table 3).

In this study, the top five comments from each video were analyzed to gain a deeper understanding of public perceptions and concerns regarding these HPV vaccination videos. The analysis revealed several significant issues: One prominent theme was the lack of clarity regarding the appropriateness of vaccinating individuals already diagnosed as HPV-positive, highlighting a critical gap in public knowledge and inadequate information on the videos. Additionally, concerns were raised about the necessity and benefits of HPV vaccination for individuals besides reproductive-age females, indicating uncertainty about the vaccine’s broader applicability in the comment section of the videos. Notably, questions regarding the efficacy and advisability of HPV vaccination for individuals outside the recommended age range were frequently observed in the comment section, underscoring the need for clearer guidance on this issue.

Another recurring issue in the comment section was the insufficient awareness of the vaccine’s protective role against genital warts, highlighting the need for more comprehensive content in the videos about the vaccine’s benefits. Financial concerns were also frequently expressed, with many comments highlighting the cost of the vaccine as a barrier to access, underscoring the importance of improving affordability and accessibility. 

The importance of family physicians in educating the public about HPV vaccination was emphasized in several comments, which identified HCPs as trusted sources of information. These findings highlight the necessity of addressing public knowledge gaps, improving vaccine accessibility through policy interventions, and strengthening the role of HCPs in disseminating accurate and comprehensive information about HPV vaccination. This highlights the need for HCPs, family physicians in particular, to actively participate in awareness efforts and provide patients with accurate, accessible information.

Discussion

This study evaluated the content quality and educational value of Turkish-language YouTube videos on HPV vaccination. The findings highlight the influence of health-related content dissemination on social media and its implications for public health. By employing multiple validated evaluation tools (PEMAT, JAMA score, VIQI, GQS, VPI), we were able to assess the video’s understandability, actionability, and quality. The results suggest that YouTube serves as a crucial platform for health communication; however, the scientific accuracy and educational value of its content require improvement.

Considering that YouTube users typically view only the first few pages of search results and that previous research indicates decreased viewer engagement with longer videos, both factors were incorporated into the study design (17). For this reason, the video duration was restricted to between 15 seconds and 15 minutes. The duration filter may have increased the likelihood of videos being watched fully, potentially enhancing the generalizability of the findings (16).

Patients’ tendency to seek health information online, along with the inconsistent quality of available content, can lead to misinformation and potentially harm the patient-physician relationship. Keelan et al. (19) conducted the first study evaluating the quality of YouTube videos on immunization. Since then, numerous studies have raised concerns about the quality and accuracy of medical information on video-sharing platforms. 

In line with these concerns, our study found a median GQS score of 3, indicating that the quality of HPV vaccine information on YouTube is moderate. Similar results have been reported for other medical topics. For instance, Radonjic et al. (20) assessed videos related to abdominal aortic aneurysms on YouTube and concluded that most were of poor quality and unreliable. They emphasize the need for HCPs to recognize the influence of such platforms and understand the nature of the information disseminated there. Likewise, Küçükakkaş et al. (21) found moderate GQS scores in YouTube videos on lymphedema rehabilitation, with university-produced videos achieving the highest mean GQS score of 3.5. 

Dirican et al. (22) evaluated English-language HPV videos on YouTube, reporting low GQS scores for misleading videos (mean = 1.84) and moderate scores for those with insufficient information (mean = 3.13). These findings suggest that the inadequacy in content quality is not limited to language-specific factors, but rather reflects a broader, global issue concerning the quality of digital health information. Notably, our dataset did not include any university-affiliated content creators; we hypothesize that their inclusion could have increased the overall mean score. 

Conversely, some studies have reported higher-quality content. For example, nearly half of the videos on self-administering subcutaneous anti-tumor necrosis factor injections and exercises for ankylosing spondylitis were rated as useful or high quality (23). Differences in findings across the studies may be attributed to variations in the health topics examined, the subjective nature of video evaluations despite standardized criteria, sample size disparities, and the potential influence of language on results. 

In the GQS system, the overall quality rating is derived by assessing content subheadings collectively, including factors like information accuracy, video flow, quality, and usefulness. In contrast, the VIQI system evaluates these subheadings separately on a scale of 1 to 5, with the total quality score calculated afterward (24). Consequently, we opted to utilize both scales in this study, and the results showed consistency. A notable observation from our study is that although most speakers were physicians, most videos were classified as low quality according to the GQS. This assessment was consistent across the other two evaluation scales as well.

In the present study, the median JAMA score is 2/4, which shows low transparency & reliability. The results suggest that the quality of Turkish information on HPV vaccines obtained from YouTube is inadequate, and users are provided with unverified information. Kunze et al. (25) found that videos related to rotator cuff disease had low JAMA and GQS scores, indicating that the videos provided unreliable and low-quality information, aligning with our findings. This could negatively impact viewers’ motivation to seek treatment and their expectations of outcomes. The low average JAMA and GQS scores in our study align with findings from other studies investigating the quality of health-related videos on YouTube, none of which reported average scores exceeding 4 (26).

A notable finding in the present study is the absence of a significant relationship between video popularity (VPI) and other evaluation metrics. In a study of YouTube videos on fibromyalgia, Ozsoy-Unubol and Alanbay-Yagci (27) found that the JAMA score had a significantly weak correlation with the number of likes and views. However, some studies contradict this finding. For example, the study of pterygium surgery by Ozturkmen and Berhuni (28) revealed a significant positive correlation between the JAMA score and the number of likes, suggesting that videos of higher quality tended to receive more engagement and positive feedback.

The phimosis study by Cilio et al. (29) also reported poor PEMAT scores, similar to those presented in the current study. Overall, the findings of our study align with the broader literature. The relatively higher actionability scores observed in the presented analysis may be attributed to the nature of our topic, which focuses on vaccination—a subject inherently requiring actionable steps. Nonetheless, the results indicate that YouTube remains an inadequate platform for effective patient education, emphasizing the need for more reliable and structured sources of health information.

In the present study, 98.75% of the speakers in the videos were physicians (86.4% gynecologists, 6.2% general surgeons, 4.9% urologists, and 2.5% others). None of them was a family physician. However, both young people and their parents report that family physicians play a crucial role in immunization by building long-term, trusting relationships with their patients. The recommendations and evidence-based information provided by family physicians about HPV vaccination significantly impact their vaccination behavior (30). In the literature, there are videos uploaded mostly by HCPs that are compatible with our research (10). Another study on English-language YouTube videos about HPV found that most of the producers were gynecologists (33%), similar to our results. However, it was followed by general practitioners (22%), which contradicts our study (16). Gynecologists mainly focus on cervical cancer and produce more YouTube content on HPV vaccination, limiting the emphasis on gender-neutral HPV vaccination.

The analysis of top comments from HPV vaccination videos revealed significant public concerns and knowledge gaps. Key issues included uncertainty about vaccinating HPV-positive individuals, doubts about the vaccine’s applicability for non-high-risk groups and older age ranges, insufficient awareness of its protective role against genital warts, and financial barriers limiting access. The current literature also suggests that parents and young people throughout Europe require information regarding the age range for the HPV vaccination, the number of required doses, and its effectiveness against various HPV-related diseases. Additionally, the cost of the vaccine, which can be a significant financial burden for families, combined with its absence from official national vaccination programs, presents a significant barrier to access, consistent with the findings revealed from the comments section of the YouTube videos in the present study (10,31). Comments also highlighted the critical role of family physicians in educating the public and addressing misinformation. These findings emphasize the need for targeted educational efforts, improved vaccine affordability, and active involvement of HCPs in raising awareness.

This study has several limitations. A small sample size may lead to a beta error, limiting the study’s statistical power. Videos with zero likes result in a VPI score of 0 regardless of view count, potentially misrepresenting the popularity of these videos. Additionally, the study did not examine the effect of the number of account followers on views and likes, as it focused on the content of videos accessible through search. This may also have affected the popularity score. However, VPI was preferred by the researchers as it is identified as the most valid popularity index for audiovisual materials in the literature. Designing more detailed and multivariate popularity scoring scales in the future may help obtain more accurate results in studies examining social media content. Another noteworthy limitation is the absence of a systematic analytical approach, such as thematic analysis, in the examination of the comments. To gain a deeper understanding of societal perceptions and to analyze social media commentary in greater depth, future studies are encouraged to adopt more comprehensive qualitative research designs. 

The literature indicates that many studies, not limited to those on HPV vaccines, have evaluated YouTube videos addressing various diseases and treatments. These studies consistently identify the prevalence of low-quality and potentially misleading content on the platform. They emphasize the critical responsibility of HCPs, particularly family physicians, to address this issue. As primary care providers, family physicians play a pivotal role in guiding individuals toward reliable health resources and combating misinformation. To improve the quality of health information on YouTube, family physicians and other HCPs should actively produce and disseminate accurate, evidence-based, and up-to-date content. In practice, this means family physicians collaborating with peers and partners to produce short myth-busting videos, interacting with the public through social media, and advocating for broader support in combating misinformation. Evidence suggests that professional involvement on social media platforms can improve content accuracy and increase community awareness. By leveraging their unique position within communities, family physicians can contribute to improved public health literacy and ensure access to trustworthy online health information.