“Headache” Online Information: An Evaluation of Readability, Quality, Credibility, and Content



 

Abdulrahman Bukhari, MBBS1; Saad Alshihri, MBBS1; Mohammed Abualenain, MBBS1; Jordan Barton, BA1; Genevieve Kupsky, BS1; Jesse M Pines, MD1; Ali Pourmand, MD, MPH, RDMS1

Perm J 2021;25:20.185

https://doi.org/10.7812/TPP/20.185
E-pub: 03/01/2021

ABSTRACT

Introduction: Headache is experienced by more than half of the world population each year. In this study, we evaluate the content, quality, and health literacy required to understand online information for patients with headaches.

Methods: We selected 4 commonly used search engines (Google, Yahoo, Bing, and Ask.com) and searched using the term “headache.” The 30 top hits on each site were selected for review. After exclusions, we examined the websites for completeness of content, readability, credibility, and quality.

Results: A total of 28 websites were included. None of the websites met our criteria for completeness. Using 2 standard measures of readability, most websites required reading skills at the 10th-grade level or greater. Only 4 of the 28 websites were readable below the eighth-grade level. Only 3 websites fulfilled all 4 credibility criteria of authorship, currency, citations, and disclosure. Most websites did not list authorship, and only 17% reported disclosures of conflicts of interest. When assessing quality of treatment information using the DISCERN tool, scores ranged from 23 to 59, with a mean score of 41, which could be interpreted as “fair” quality.

Conclusions: We found variable content and quality in online headache websites for patients. Many of these websites failed to disclose information about authorship, conflicts of interest, and details on the prognosis or prevention of headaches. Readability, credibility, completeness, and quality of information were lacking in most websites.

INTRODUCTION

Headache is one of the most common neurologic disorders worldwide. According to the World Health Organization, almost half of the adult population has had a headache at least one time during the past year.1 The most common types of primary headaches are tension-type headaches, migraines, and cluster headaches.2 Secondary headaches—originating from alternative causes—may be the result of serious underlying diseases such as subarachnoid hemorrhage, intracranial tumors, or meningitis.

Despite its prevalence, headache accounts for just 5% of all general practice consultations,3 and only 41% of people with chronic migraines have consulted a health-care provider about their condition.4 With increasing frequency, patients are turning to the internet as a source of health information. In one study, nearly 59% of adult Americans reported searching for health information online.5 The discrepancy in the amount of consultations and the amount of internet searches suggests that individuals are seeking information on headache from the internet rather than from health-care providers.

Internet users searching “headache” online encounter a variety of websites, including personal blogs, education portals, peer-reviewed articles, and patient information from professional societies. These sources of online information also come with varying degrees of content as well as reliability. Prior studies have shown that online health information is highly variable in content and quality, raising concerns about the accuracy and trustworthiness of consumer-oriented websites.6,7

To be effective, online information should be clear, accessible, and consistent with patient education materials provided by clinicians. Two important components of this are required health literacy and readability. Health literacy is the ability of individuals and communities to obtain and understand basic health information and services needed to make appropriate health decisions.8,9 Readability is the ease of reading a written text.10 To our knowledge, no studies have examined the content, quality, or health literacy required to read patient-oriented websites on headache. In this study, we aimed to describe the content in patient websites for headache, as well as the required health literacy, readability, and quality.

METHODS

In this study, we used 4 popular search engines used (Google, Yahoo, Bing, and Ask.com) to identify patient-oriented headache websites to examine the content, readability, and quality of the material. We searched the term “headache” across these 4 search engines and examined the first 30 results from each engine. We limited our websites to the top 30 results because prior studies have found that about 91% of internet traffic in an average search is concentrated on the first page.11

Inclusion and Exclusion Criteria

We identified the 30 top websites each from the 4 search engines, for a total of 120 websites. Websites that consisted of scientific research articles only, those that did not provide information about headache, advertisements, sponsored websites, medical dictionaries, websites that provided information on only 1 type of headache, and duplicate sites within and across the search engines were excluded. After applying exclusion criteria, 28 websites were evaluated. For each of the following scales, we tabulated the data using descriptive statistics. This study was determined to be not human subject research and did not warrant an institutional review board review.

Content Assessment

Our team used 11 criteria (description, types, causes, associated symptoms, triggers, red-flag symptoms, diagnosis, treatment, alternative treatment, prevention, and prognosis) to quantify completeness of information for each website. These criteria were developed based on the team’s assessment of the information about headache that websites for patients should contain. An initial list was developed by the study team, and was then refined based on feedback from a group of 18 physicians.

Readability Assessment

The readability of websites was assessed using the SMOG index tool and the Flesch-Kincaid tool. Both tools estimate the years of education needed to comprehend an article. For example, a readability score of 7 means an average student in the sixth grade can understand the text. The Flesch-Kincaid tool measures the average number of syllables per word and the number of words per sentence, whereas the SMOG index tool uses a formula developed by McLaughlin, which generates a score based on the number of words with 3 or more syllables at the beginning, middle, and end of a passage.12

Credibility Assessment

The JAMA benchmark tool consists of 4 criteria (authorship, attribution, disclosure, and currency) developed by the Journal of the American Medical Association to assess credibility. Authorship refers to mention of the authors’ names, affiliations, and credentials. Attribution ensures that references are mentioned appropriately. Disclosure identifies whether the ownership and any conflict of interest are properly declared. Last, currency establishes whether the content and the resources are properly dated and updated. Each website is given a score of 1 for each element that is present and a 0 for each element that is absent.

Quality Assessment

The DISCERN instrument is a questionnaire used to assess the quality of health information for consumers.13 It is comprised of approximately 16 questions, with a rating scale for each question of 1 to 5, where 1 means the article did not fulfill any elements of the question and 5 means the article fulfilled all elements of the question. Questions 1 through 8 assess reliability and whether the sources of information can be trusted, questions 9 through 15 focus on specific details of the information about the treatment choices, and question 16 assesses the overall quality of information. The total DISCERN score is calculated by summing the scores, with scores that can range from 15 to 80.

RESULTS

Content Assessment

Across the 28 included websites, none fulfilled all 11 content criteria in our checklist. Only 7 (25%) websites fulfilled 10 of the 11 criteria. Only a minority of websites had information on prevention and prognosis, with 12 (43%) and 4 (14%) containing that information, respectively. Most websites described the causes and types of headaches, with 27 (96%) and 24 (85%) providing that information, respectively (Figure 1 and Table 1).

tpj20185f1

Figure 1. Percentage of websites for each checklist item.

Table 1. Number and percentage of websites for each checklist item

Checklist items No. of websites Percentage of websites
Causes 27 96
Types 24 86
Triggers 22 79
Associated symptoms 21 75
Red-flag symptoms 21 75
Treatment 21 75
Alternative treatment 20 71
Diagnosis 19 68
Description 18 64
Prevention 12 43
Prognosis 4 14

Health Literacy and Readability

The readability scores of online health information about headache ranged from 5.6 to 17.3 using the SMOG tool, with a mean of 10.2. Readability scores using the Flesch-Kincaid tool ranged from 6.3 to 22.1, with a mean of 11.5. Values generated using these tools can be interpreted as, to understand most of the material, one needs the reading ability of at least a 10th grader. Only 4 of the 28 websites had a readability score less than the eighth-grade level.

Reliability

The reliability of headache-related websites was assessed using the JAMA Benchmark Criteria (authorship, currency, attribution, and disclosure); only 3 websites fulfilled all 4 criteria. The majority of the websites (n = 23, 82%) provided the date on which the material was posted or updated. Authorship information was provided in 17 websites (61%); attribution and disclosure were the least common criteria noted, mentioned in 14 (54%) and 11 (39%) websites, respectively.

Quality

The quality of the headache-related websites was assessed using the DISCERN tool, with an emphasis on treatment. Websites were rated as “excellent” (63–80), “good” (51–62), “fair” (39–50), “poor” (27–38), and “very poor” (15–26).14-16 The mean score was 41 (“fair”), with a range of 23 to 59. Only 7 websites were classified as “good,” and 3 were “very poor.” None of the websites fell in the excellent range (Figure 2 and Table 2).

tpj20185f2

Figure 2. Average DISCERN score for each item.

Table 2. Total DISCERN score for each website

Websites Total DISCERN score
www.drugs.com/cg/acute-headache.html 59
www.webmd.com/migraines-headaches/migraines-headaches-basics#1 56
www.buoyhealth.com/current/23-headache-remedies-comprehensive-list/ 56
www.emedicinehealth.com/mild_headache/article_em.htm 55
https://en.wikipedia.org/wiki/Headache 55
www.medicalnewstoday.com/articles/320767.php 54
https://familydoctor.org/condition/headaches/ 52
https://medlineplus.gov/headache.html 50
www.migrainetrust.org/about-migraine/types-of-migraine/other-headache-disorders/headache/ 49
www.medicinenet.com/headache/article.htm 48
www.healthline.com/health/headache 47
www.wikem.org/wiki/Headache 46
https://patient.info/brain-nerves/headache-leaflet 45
https://my.clevelandclinic.org/health/diseases/9639-headaches-in-adults 39
www.ninds.nih.gov/Disorders/All-Disorders/Headache-Information-Page 39
emedicine.medscape.com/article/1048596-overview 39
www.health.com/health/condition-article/0,,20327041,00.html 38
www.hopkinsmedicine.org/health/conditions-and-diseases/headache 37
www.mayoclinic.org/symptoms/headache/basics/definition/sym-20050800 34
www.radiologyinfo.org/en/info.cfm?pg=headache 34
www.nhsinform.scot/illnesses-and-conditions/brain-nerves-and-spinal-cord/headaches 34
www.merckmanuals.com/professional/neurologic-disorders/headache/approach-to-the-patient-with-headache 33
http://healthlibrary.brighamandwomens.org/Search/85,P00784 32
https://kidshealth.org/en/kids/headache.html 31
www.health.harvard.edu/pain/headache-when-to-worry-what-to-do 27
www.sciencedaily.com/terms/headache.htm 23
http://chemocare.com/chemotherapy/side-effects/headache.aspx 23
www.who.int/news-room/fact-sheets/detail/headache-disorders 23

DISCUSSION

In this study, we identified 28 websites that patients commonly encounter when searching for online information about headache and found that the type of content most frequently included were causes of headache, followed by types and triggers. Information about associated symptoms, available treatment options, and alternative treatment were observed in more than 50% of the websites. By contrast, information regarding prevention strategies and prognosis was present in a lower proportion of websites. Therefore, patients may read some websites that may not answer all their potential questions about headache and may have to search multiple sites for answers.

The average readability of the websites was well above the recommended level of sixth to eighth grade.17 This could be attributed to the fact that health-related information contains many polysyllabic terms and lengthy sentences. Another contributing factor may be that these websites are mostly written by physicians who may not trained to write in simple terms for a lay audience. Writers of websites for patients should consider shortening sentences and using simpler terms in order to be accessible to all patients. In addition, an alternative solution might be to use more visual media or diagrams, which may help both low- and high-literacy individuals both absorb and understand the information.

When assessing the reliability of the websites, we found that that currency was the most reported element, with around 82% of sites providing dates for when the content was posted or updated. Authorship information was present in only 60% of websites. Authorship can add assurance and authenticity to the material, enabling readers to know who wrote the content. Attribution was only included in approximately half of the websites, which allows readers to know the source of the information, can increase confidence in the material, as well as provide additional literature sources. Disclosure of possible conflicts of interest was reported the least and was only mentioned in 39% of sites, which may have an impact on the overall balance and bias of information.

When it came to quality, the average website was “fair,” mostly because of the absence of clear aims. Lack of a clear aim can make it difficult for readers to know whether the website can answer particular questions or whether it is applicable to their problem. Furthermore, most websites did not emphasize the importance of treating headaches, including how headaches can affect daily life and the consequences of not treating headaches.

Based on the checklist assessment of the 28 websites, the top 3 websites in terms of content completeness (fulfilling 10 of the 11 checklist items) were www.medicinenet.com, www.emedinicehealth.com, and www.drugs.com. The websites with the lowest checklist scores (4 of 11) included www.sciencedaily.com, www.nhsinform.scot, and www.who.int (Table 3).

Table 3. Checklist score for each website

Websites Checklist score
www.medicinenet.com/headache/article.htm 10
www.emedicinehealth.com/mild_headache/article_em.htm 10
www.drugs.com/cg/acute-headache.html 10
http://healthlibrary.brighamandwomens.org/Search/85,P00784 10
www.hopkinsmedicine.org/health/conditions-and-diseases/headache 10
https://en.wikipedia.org/wiki/Headache 10
https://familydoctor.org/condition/headaches/ 10
www.webmd.com/migraines-headaches/migraines-headaches-basics#1 9
www.health.harvard.edu/pain/headache-when-to-worry-what-to-do 9
www.healthline.com/health/headache 9
www.migrainetrust.org/about-migraine/types-of-migraine/other-headache-disorders/headache/ 9
www.medicalnewstoday.com/articles/320767.php 8
www.wikem.org/wiki/Headache 8
https://my.clevelandclinic.org/health/diseases/9639-headaches-in-adults 7
www.health.com/health/condition-article/0,,20327041,00.html 7
emedicine.medscape.com/article/1048596-overview 7
www.merckmanuals.com/professional/neurologic-disorders/headache/approach-to-the-patient-with-headache 7
www.buoyhealth.com/current/23-headache-remedies-comprehensive-list/ 7
www.radiologyinfo.org/en/info.cfm?pg=headache 6
www.ninds.nih.gov/Disorders/All-Disorders/Headache-Information-Page 6
http://chemocare.com/chemotherapy/side-effects/headache.aspx 6
https://patient.info/brain-nerves/headache-leaflet 6
www.mayoclinic.org/symptoms/headache/basics/definition/sym-20050800 5
https://medlineplus.gov/headache.html 5
https://kidshealth.org/en/kids/headache.html 5
www.sciencedaily.com/terms/headache.htm 4
www.nhsinform.scot/illnesses-and-conditions/brain-nerves-and-spinal-cord/headaches 4
www.who.int/news-room/fact-sheets/detail/headache-disorders 4

LIMITATIONS

This study has several limitations. We only evaluated 28 websites, and it is possible that our results would be different if we evaluated additional or different websites. Also, we only evaluated websites written in English. There is some degree of subjectivity involved when using tools like the JAMA Benchmark and DISCERN instruments. Also, there was also subjectivity in assessing content using an internally generated checklist. Scores generated using the SMOG and Flesch-Kincaid readability assessment tools were computed objectively for written text. Nevertheless, these tools do not assess other components of websites such as photos or videos, both of which can impact patient understanding. Furthermore, these tools do not take into consideration the complexity of medical terminology or reader familiarity with such words. Because the content and popularity of websites found online is ever-changing, our results cannot be generalized beyond the date of publication.

CONCLUSIONS

Our study showed that health information on headache had low readability, mixed credibility, fair quality, and variable content. In general, most websites did not provide information on prognosis or prevention of headaches. Measures should be taken to ensure readers have access to reliable information online.

Disclosure Statement

The author(s) have no conflicts of interest to disclose.

Author Affiliations

1Emergency Medicine Department, George Washington University School of Medicine and Health Sciences, Washington, DC

Corresponding Author

Ali Pourmand, MD, MPH, RDMS ()

Author Contributions

Abdulrahman Bukhari, MBBS; Saad Alshihri, MBBS; Mohammed Abualenain, MBBS; Jordan Barton, BA; Ali Pourmand, MD, MPH, RDMS; and Jesse M Pines, MD; designed the study. Abdulrahman Bukhari, MBBS; Saad Alshihri, MBBS; Mohammed Abualenain, MBBS; Jordan Barton, BA; and Genevieve Kupsky, BS; performed the analysis, drafted the manuscript, and designed the tables and figures. All authors discussed the results and contributed to the final manuscript. Ali Pourmand, MD, MPH, RDMS, and Jesse M Pines, MD, supervised the project.

Financial Support

This is a nonfunded study, with no compensation or honoraria for conducting the study.

References

1. Pietrasik T. Headache disorders. World Health Organization; 2020. Accessed January 12, 2020. www.who.int/news-room/fact-sheets/detail/headache-disorders

2. Olesen J, Göbel H. The international classification of headache disorders 3rd edition.IHS Classification ICHD-3. International Headache Society; 2019. Accessed January 12, 2020. https://ichd-3.org/

3. Frese T, Druckrey H, Sandholzer H. Headache in general practice: Frequency, management, and results of encounter. Int Sch Res Notices 2014 Oct 29;2014:1–6. DOI: https://doi.org/10.1155/2014/169428

4. Dodick DW. Migraine. Lancet 2018 Mar;391(10127):1315–30. DOI: https://doi.org/10.1016/s0140-6736(18)30478-1

5. Fox S. Health topics. Pew Research Center: Internet, Science & Tech: 2019. Accessed January 12, 2020. www.pewresearch.org/internet/2011/02/01/health-topics-2/

6. Eysenbach G, Powell J, Kuss O, Sa ER. Empirical studies assessing the quality of health information for consumers on the world wide web: A systematic review. J Am Med Assoc 2002 May;287(20):2691. DOI: https://doi.org/10.1001/jama.287.20.2691, PMID:12020305.

7. Friedman DB, Hoffman-Goetz L. A systematic review of readability and comprehension instruments used for print and web-based cancer information. Health Educ Behav 2006 Jun;33(3):352–73. DOI: https://doi.org/10.1177/1090198105277329

8. Nielson-Bohlman L, Panzer A, Kindig D, eds. Health literacy: A prescription to end confusion. Washington, DC: National Academies Press; 2004.

9. World Health Organization. Track 2: Health literacy and health behaviour. World Health Organization; 2010. Accessed January 12, 2020. www.who.int/healthpromotion/conferences/7gchp/track2/en/

10. DuBay W. The principles of readability. Costa Mesa, CA: Impact Information; 2004.

11. Chitika, Inc. The value of Google result positioning: Chitika Online Advertising Network. Chitika, Inc; 2013. Accessed January 12, 2020. https://web.archive.org/web/20130610125944/https://chitika.com/google-positioning-value

12. The SMOG readability formula, a simple measure of Gobbledygook. 2020. Accessed January 12, 2020. https://readabilityformulas.com/smog-readability-formula.php

13. General instructions: The DISCERN instrument. Accessed January 12, 2020. www.discern.org.uk/general_instructions.php

14. Weiss BD. Health literacy: A manual for clinicians. Chicago, IL: American Medical Association, American Medical Foundation; 2003.

15. Cotugna N, Vickery CE, Carpenter-Haefele KM. Evaluation of literacy level of patient education pages in health-related journals. J Community Health 2005 Jun;30(3):213–9. DOI: https://doi.org/10.1007/s10900-004-1959-x, PMID:15847246.

16. National Institutes of Health. How to write easy to read health materials: National Library of Medicine website. Accessed February 1, 2009. www.nlm.nih.gov/medlineplus/etr.html

17. Eltorai AE, Ghanian S, Adams CA Jr, Born CT, Daniels AH. Readability of patient education materials on the American Association for Surgery of Trauma website. Arch Trauma Res 2014 Jun;3(2):e18161. DOI: https://doi.org/10.5812/atr.18161, PMID:25147778.

Keywords: credibility, headache, online, quality, readability

ETOC

Click here to join the eTOC list or text ETOC to 22828. You will receive an email notice with the Table of Contents of The Permanente Journal.

CIRCULATION

2 million page views of TPJ articles in PubMed from a broad international readership.

Indexing

Indexed in MEDLINE, PubMed Central, EMBASE, EBSCO Academic Search Complete, CrossRef, and SciVerse/Scopus.


                                             

 

 

ISSN 1552-5775 Copyright © 2020 thepermanentejournal.org

All Rights Reserved