Evaluation of Quality and Readability of Health Information Websites Identified through India’s Major Search Engines

Raj, S.; Sharma, V. L.; Singh, A. J.; Goel, S.

doi:https://doi.org/10.1155/2016/4815285

Advances in Preventive Medicine

On this page

Abstract Introduction Methods Results Discussion Conclusions Additional Points Disclosure Acknowledgments References Copyright Related Articles

Research Article | Open Access

Volume 2016 | Article ID 4815285 | https://doi.org/10.1155/2016/4815285

Evaluation of Quality and Readability of Health Information Websites Identified through India’s Major Search Engines

S. Raj,¹V. L. Sharma,¹A. J. Singh,²and S. Goel²

Academic Editor: Gerardo E. Guillén Nieto

Received22 Sept 2015

Revised09 Feb 2016

Accepted02 Mar 2016

Published28 Mar 2016

Abstract

Background. The available health information on websites should be reliable and accurate in order to make informed decisions by community. This study was done to assess the quality and readability of health information websites on World Wide Web in India. Methods. This cross-sectional study was carried out in June 2014. The key words “Health” and “Information” were used on search engines “Google” and “Yahoo.” Out of 50 websites (25 from each search engines), after exclusion, 32 websites were evaluated. LIDA tool was used to assess the quality whereas the readability was assessed using Flesch Reading Ease Score (FRES), Flesch-Kincaid Grade Level (FKGL), and SMOG. Results. Forty percent of websites () were sponsored by government. Health On the Net Code of Conduct (HONcode) certification was present on 50% () of websites. The mean LIDA score (74.31) was average. Only 3 websites scored high on LIDA score. Only five had readability scores at recommended sixth-grade level. Conclusion. Most health information websites had average quality especially in terms of usability and reliability and were written at high readability levels. Efforts are needed to develop the health information websites which can help general population in informed decision making.

1. Introduction

Health promotion recognizes the vital importance of access of people to information. This is critical in achieving effective participation and empowerment of people and communities in health related activities [1]. Due to revolution in information and communication technology, the Internet has now become important source of health information both for consumers and for providers of health information [2–6]. Everything is now available on the click of a button. The developing countries like India have registered a year-on-year growth of 32% in number of Internet users. According to Internet and Mobile Association of India (IAMAI), India had more Internet users than United States by December 2014 [7]. According to Online Consumer Panel Report, 2011, 72% of Indian respondents used the Internet to access healthcare related information [8].

However, in order to make informed decisions regarding health, the available information should be reliable and accurate. On the Internet, due to freedom of information, almost anyone can create a website and offer expert advice regarding a host of topics. Research has shown that most contents on health information websites are not authored by medical professionals and not policed by any governing body or adhered to any ethical regulations [9]. Studies have shown that the quality of available health information on World Wide Web is not reliable [10–15]. Therefore, there is a risk that health information available may be misleading or dangerous [16]. Such information of dubious quality can do more harm than good [17]. Due to these rising concerns over the quality of health information available on the Internet, the Health On the Net foundation (HON Foundation) issued a code of conduct (HONcode) for health and medical information websites to address their reliability and usefulness. HONcode certification is an ethical standard aimed at offering quality, objectivity, and transparency of health information on websites by setting up a minimum set of principles. It is purely voluntary and demonstrates the intent of a website to publish transparent information [18].

Even if the contents of the websites are reliable, another important factor is the ability of users to understand those materials [19]. The available health information should be at the level which is easier to understand by the public and is not lost behind medical vocabulary. Readability of a written text is an objective measure of the reading skills an individual must possess to understand that material [20]. So, the present study was conducted to assess the quality and readability of health information websites on World Wide Web.

2. Methods

This cross-sectional study was carried out in month of June, 2014. The key words “Health” and “Information” were typed on two search engines namely “Google” and “Yahoo.” These search engines were taken as they are the top most search engine of the year 2014 [21]. In 2015, Yahoo Inc. formally entered a service agreement with search engine giant Google, Inc., where Google will provide Yahoo with search advertisements through Google’s AdSense for Search (AFS) service, web algorithmic search services through Google’s web search service, and image search services. By this agreement these two have collectively became the top search engine for accessing information [22]. Secondly, literature has shown that Google is the major search engine used to get health information [23].

There are few reports which documented that consumers often visit fewer than 25 topmost links found on a search, with most links being in the top five rank of the search results [24]. The searches were made through an India based IP address so as to get the most accessible sites in context to the study settings to make relevant contextual policy implications.

Therefore, we analyzed the top 25 web links per engine for their quality and readability. The criteria for inclusion of health information sites were that it should be written in English and offer general health information about various diseases. The repetitions, blogs, discussion groups, and specific journal article links were excluded. Out of 50 websites on health information (25 from each of the search engines), after exclusion, 32 websites were evaluated as follows. The presence of HONcode certification on the evaluated websites was also noted.

List of websites evaluated for quality and readability in the study (arranged in alphabetical order) is as follows: http://www.aarogya.com/ http://www.cdc.gov/ http://www.clevelandclinic.org/ http://familydoctor.org/ http://hardinmd.lib.uiowa.edu/ http://www.health.gov/ http://www.health.nih.gov/ http://www.healthcareguide.com/ http://www.healthfinder.org/ http://www.healthline.com/ http://www.healthyindia.org/ http://www.heathlibrary.com/ http://www.hifa2015.org/ http://www.indmedica.com/ http://www.intelehealth.com/ http://www.mayoclinic.org/ http://www.medicinenet.com/ http://www.medindia.net/ http://www.medlineplus.com/ http://www.mohfw.nic.in/ http://www.motherchildtrust.org/ http://www.netwellness.org/ http://www.nhp.gov.in/ http://www.nlm.nih.gov/ http://www.nrhm.gov.in/ http://www.onlymyhealth.com/ http://www.patient.co.uk/health/ http://www.thehealthsite.com/ http://www.union-imdp.org/ http://www.webMD.com/ http://www.who.int/ http://www.360living.in/

2.1. Quality

The LIDA Instrument (The LIDA Instrument, Version 1.2, 2007, Minervation Ltd., Oxford, UK) was used to assess the quality of these websites. It is a validated method of evaluating healthcare websites based on three important areas: accessibility, usability, and reliability [25].

2.1.1. Accessibility

It is based on ease of accessibility of the websites. The websites should meet legal accessibility requirements, without restrictions and outdated HTML code. The maximum possible score for accessibility was 54.

2.1.2. Usability

It depends upon the understandability of the content in the websites. The aspects assessed included clarity of presentation, consistency of web page design, functionality including intuitive browsing and search facilities, and engageability. The maximum possible score was 12.

2.1.3. Reliability

It is based on the accuracy of the information on the websites. It includes regular updates, clear declaration of conflict of interests, rigorous methodology for content production, and output. The maximum possible score was 30.

The maximum total score of a website on LIDA tool was 96. The accessibility score was computed by filling in the web address of the site on a customized web platform (http://www.minervation.com/mod product/LIDA). A nine-item questionnaire was applied to evaluate usability and reliability. The responses were graded from 0 to 3 (0: never; 1: sometimes; 2: mostly; 3: always). The scores lower than 50% were low, between 50 and 90% were medium, and equal to and above 90% were taken as high.

2.2. Readability

Three readability formulas were used to assess the readability of the websites.

The Flesch Reading Ease Score (FRES) is as follows [26]:where RE is readability ease; ASL is average sentence length (i.e., the number of words divided by the number of sentences); and ASW is average number of syllables per word (i.e., the number of syllables divided by the number of words).

The output score ranges from 0 to 100. The scores between 90.0 and 100.0 are considered easily understandable by an average 5th grader; scores between 60.0 and 70.0 are considered easily understood by 8th and 9th graders; scores between 0.0 and 30.0 are considered easily understood by college graduates. The higher the number is, the easier the text is to read.

The Flesch-Kincaid Grade Level (FKGL) is as follows [27]:where FKRA is Flesch-Kincaid reading age; ASL is average sentence length (i.e., the number of words divided by the number of sentences); ASW is average number of syllables per word (i.e., the number of syllables divided by the number of words).

FKRA score of 5 indicates that the available health information can be comprehended by fifth grade school level and a score of 9.3 means that a ninth grader would be able to read the document.

Simple Measure of Gobbledygook (SMOG) is as follows [28]: SMOG Readability Formula estimates the years of education a person needs to understand a piece of writing.

To calculate these readability scores, the online Readability Text Consensus Tool was used [29]. This tool analyzes the text and calculates the number of sentences, words, syllables, and characters in the sample. A sample of three, 50 to 100-words was copied from the beginning, the middle, and the end of the page of the website, respectively, and was pasted into an online readability calculator program to check the readability scores. Accuracy of the online method has been confirmed by the prior comparison of automated and manual calculation [30]. Data entry and analysis were done using IBM Statistical Package for Survey Solutions version 16 (SPSS-16). value of less than 0.05 was considered significant.

3. Results

Out of total of 32 health information websites assessed, 40% () were sponsored by government and 35% () were sponsored by commercial agencies. Health On the Net Code of Conduct (HONcode) certification was present on 50% () websites (Table 1).

3.1. Quality

The mean LIDA score for accessibility was 47.2 (66.7%) out of possible 54. Eleven (34.4%) websites had high accessibility score (more than 90%). The average usability score was 8 (66.6%) out of total 12. None of the websites scored high on this parameter. The mean reliability score was 19 (63.3%) and only 5 (15.6%) websites scored more than 90% (high) on this parameter. The overall mean LIDA score for the websites was found to be 74.3 (77.4%) with SD 9.2 and interquartile range of 55–88, bordering on an overall medium score as shown in box-and-whisker plot in Figure 1 and Table 2. Only 3 (9.4%) websites scored high on LIDA score.

3.2. Readability

The mean FRES, FKGL, and SMOG score of health information websites were found to be 50.5, 10.2, and 9.6, respectively. Less than half (46.9%) of the websites had easy to read FRES score (60–100). Only 5 (15.6%) websites had FKGL, SMOG readability score at sixth-grade reading level as recommended to enhance the quality of educational materials for public [31] (Table 3).

Mean LIDA score was found to be significantly associated with the provider of the health information websites. Among the different readability scores, mean SMOG and FKGL score was also found to be significantly associated with the intended audience of these websites (Table 4).

4. Discussion

There is a wealth of health information and resources available on the Internet. However, the quality of health information available on the Internet is variable and dubious. The wrong information on these websites can put the health of the public “at risk” and can lead to serious consequences [32].

Search engines are the popular way to find health information on the Internet. According to Harris Poll, 2011, more than two-thirds (69%) of health information queries start with common search engine [33]. Therefore, we used main two search engines, namely, Google and Yahoo, to assess the quality and readability health information websites. Following the standard retrieval technique in e-health information searches, websites were explored beyond the first page of the links [24].

The present study found that quality of health information websites on parameters of accessibility, reliability, and readability by LIDA tool was quite different and average. Similar results have been found in other studies, wherein the available information on different medical conditions on the Internet has been found to be of low quality [10, 13, 15, 16, 34, 35]. Despite the ongoing revolution in the field of health information technology, efforts are needed to provide the general public with accurate and reliable health information which can help them in informed decision making regarding their health.

The methods to determine quality of medical content on websites are variable. Over the recent years, many tools are available to review health information websites. However, many of them are of doubtful utility [36]. We chose LIDA tool as it validated and covers all aspects of quality as accessibility, usability, and reliability and checks HTML and metadata for errors. The mean LIDA score (74.3) of health information websites in our study was found to be within the range of other studies [13, 15, 34]. Among the individual categories, the websites were easily accessible but usability and reliability score were low. So, further improvement is required especially in areas like transparency of sites, authorship by experts, update frequency, and citing of relevant sources. Government websites scored better in the LIDA tool assessment for health information.

Along with the quality, it is imperative that these websites should be written at a level which can be easily comprehended by the general public. It should be easily understood and accurate health information can enhance self-esteem, greater participation in care, and better informed decision making, thus leading to empowerment of the patients and their caregivers [37]. By using three readability formulas FRES, FKGL, and SMOG, we found that the majority of the health information websites were written in language which cannot be comprehended by a recommended sixth-grade reading level. The results were consistent with other studies [11, 14]. So, even if there is increased access to the Internet, the online health information is still hard to understand by majority of the people. Therefore, the administrators of the websites should be encouraged to improve the readability of the health information content so as to communicate a broad range of messages to a wide variety of audiences.

The present study found that only half of the websites were HONcode certified. Literature has shown that HONcode certification was present in 4%–30% of the websites [38, 39]. However, our study found no significant difference in quality of scores of websites irrespective of HONcode certification. Similar results have also been seen in other studies [15, 40]. This may be because of the fact that some of the websites may be suitable for certification but have not sought HONcode certification due to lack of awareness regarding this certification as this requires a voluntary application by the website managers.

The study has few important limitations. First, in the study we used only two search engines for retrieving information and hence cannot draw more general conclusions. However, the two search engines are the topmost engines used globally for searching about health information. Secondly, we used general term “health information” for evaluation of the websites instead of diseases specific key words since this paper was intended to assess the quality of information assessed by a disease-free (or apparently healthy) individual who would use the key words as used in the present study. The results might be different if we would have taken some diseases specific key words under the search strategy. Thirdly, the results are India specific as the “first 25” links will vary from country to country depending on the server used and hence will not be reflective of the health information websites browsed in another country. The country restriction was intentionally done so as to get the most accessible sites in context to the study settings to make context specific policy implications.

5. Conclusions and Recommendations

Most health information websites have average quality especially in terms of usability and reliability and are written at readability levels above the recommended sixth-grade reading level. Efforts are needed to develop the health information websites which can help general population in informed decision making.

Additional Points

(i)This is the first study to assess both the readability and quality of general health information websites.(ii)Study indicates that websites are not meeting the recommended standards for readability and quality and thus raises concerns about their effectiveness in educating public.

Disclosure

Work is attributed to Centre for Public Health, Panjab University.

Competing Interests

The authors declare that they have no competing interests.

Acknowledgments

The authors acknowledge the funding support by University Grants Commission, India, as Junior Research Fellowship to the Principal Author.

References

World Health Organization, The World Health Report 2008: Primary Health Care Now More Than Ever, WHO, Geneva, Switzerland, 2008, http://www.who.int/whr/2008/en/.
D. Lacher, E. Nelson, W. Bylsma, and R. Spena, “Computer use and needs of internists: a survey of members of the American College of Physicians-American Society of Internal Medicine,” in Proceedings of the AMIA Symposium, pp. 453–456, 2000.
View at: Google Scholar
S. Fox and L. Rainie, The Online Health Care Revolution: How the Web Helps Americans Take Better Care of Themselves, Pew Charitable Trusts, Washington, DC, USA, 2000.
N. L. Bennett, L. L. Casebeer, R. E. Kristofco, and S. M. Strasser, “Physicians' Internet information-seeking behaviors,” Journal of Continuing Education in the Health Professions, vol. 24, no. 1, pp. 31–38, 2004.
View at: Publisher Site | Google Scholar
B. W. Hesse, D. E. Nelson, G. L. Kreps et al., “Trust and sources of health information. The impact of the internet and its implications for health care providers: findings from the first health information national trends survey,” Archives of Internal Medicine, vol. 165, no. 22, pp. 2618–2624, 2005.
View at: Publisher Site | Google Scholar
A. D. Abbas, A. M. Abubakar, B. Omeiza, and K. Minoza, “Information-seeking behavior and computer literacy among resident doctors in Maiduguri, Nigeria,” Annals of African Medicine, vol. 12, no. 2, pp. 115–119, 2013.
View at: Publisher Site | Google Scholar
India to have 243 million internet users by June 2014: IAMAI, November 2014, http://timesofindia.indiatimes.com/tech/tech-news/India-to-have-243-million-internet-users-by-June-2014-IAMAI/articleshow/29563698.cms.
S. Bhandari and A. Kapoor, “Catching the e-healthcare bug, Webchutney digital healthcare report, 2011,” 2014, http://www.slideshare.net/sidharthrao/webchutney-digital-healthcare-report-2011.
View at: Google Scholar
M. S. Eastin, “Credibility assessments of online health information: the effects of source expertise and knowledge of content,” Journal of Computer-Mediated Communication, vol. 6, no. 4, 2001.
View at: Google Scholar
P. Doupi and J. Van der Lei, “Rx medication information for the public and the WWW: quality issues,” Medical Informatics and the Internet in Medicine, vol. 24, no. 3, pp. 171–179, 1999.
View at: Publisher Site | Google Scholar
G. K. Berland, M. N. Elliott, L. S. Morales et al., “Health information on the internet: accessibility, quality, and readability in English and Spanish,” Journal of the American Medical Association, vol. 285, no. 20, pp. 2612–2621, 2001.
View at: Publisher Site | Google Scholar
G. Eysenbach, J. Powell, O. Kuss, and E.-R. Sa, “Empirical studies assessing the quality of health information for consumers on the World Wide Web: a systematic review,” The Journal of the American Medical Association, vol. 287, no. 20, pp. 2691–2700, 2002.
View at: Publisher Site | Google Scholar
A. Roshan, S. Agarwal, and R. J. A. England, “Role of information available over the internet: what are the parents of children undergoing tonsillectomy likely to find?” Annals of the Royal College of Surgeons of England, vol. 90, no. 7, pp. 601–605, 2008.
View at: Publisher Site | Google Scholar
T. Petch, Content Analysis of Selected Health Information Websites: Final Report, Simon Fraser University, Vancouver, Canada, 2004, http://www.sfu.ca/act4hlth/pub/working/Content%20Analysis.pdf.
R. Soobrah and S. K. Clark, “Your patient information website: how good is it?” Colorectal Disease, vol. 14, no. 3, pp. e90–e94, 2012.
View at: Publisher Site | Google Scholar
P. Impicciatore, C. Pandolfini, N. Casella, and M. Bonati, “Reliability of health information for the public on the World Wide Web: systematic survey of advice on managing fever in children at home,” British Medical Journal, vol. 314, no. 7098, pp. 1875–1879, 1997.
View at: Publisher Site | Google Scholar
H. Bower, “Internet sees growth of unverified health claims,” British Medical Journal, vol. 313, no. 7054, p. 381, 1996.
View at: Google Scholar
Health On the Net, “HONcode principles,” 2014, http://www.hon.ch/HONcode/Webmasters/Conduct.html.
View at: Google Scholar
J. C. Wyatt, “Commentary: measuring quality and impact of the world wide web,” The British Medical Journal, vol. 314, no. 7098, pp. 1879–1881, 1997.
View at: Publisher Site | Google Scholar
S. Badarudeen and S. Sabharwal, “Assessing readability of patient education materials: current role in orthopaedics,” Clinical Orthopaedics and Related Research, vol. 468, no. 10, pp. 2572–2580, 2010.
View at: Publisher Site | Google Scholar
World Top 50 best poular serach engines list: 2014, http://topbestlisted.blogspot.com/2011/08/worlds-top-10-best-popular-search.html.
G. McFarlane, “Google Vs. Bing Vs. Yahoo: The Search Engine Wars (MSFT, GOOG),” 2015, http://www.investopedia.com/articles/investing/070915/google-vs-bing-vs-yahoo-search-engine-wars.asp.
View at: Google Scholar
Pew Research Centre, Healt Fact Sheet, January 2014, http://www.pewinternet.org/fact-sheets/health-fact-sheet.
G. Eysenbach and C. Köhler, “How do consumers search for and appraise health information on the world wide web? Qualitative study using focus groups, usability tests, and in-depth interviews,” British Medical Journal, vol. 324, no. 7337, pp. 573–577, 2002.
View at: Publisher Site | Google Scholar
The LIDA Instrument (full version 1.2), http://www.minervation.com/lida-tool/minervation-lida-instrument-v1-2/.
R. F. Flesch, “A new readability yardstick,” Journal of Applied Psychology, vol. 32, no. 3, pp. 221–233, 1948.
View at: Publisher Site | Google Scholar
J. P. Kincaid, R. P. Fishburne, R. L. Rogers, and B. S. Chissom, “Derivation of new readability formulas (automated readability index, fog count, and flesch reading ease formula) for navy enlisted personnel,” Research Branch Report 8–75, Chief of Naval Technical Training: Naval Air Station Memphis, 1975, http://www.dtic.mil/dtic/tr/fulltext/u2/a006655.pdf.
View at: Google Scholar
G. H. McLaughlin, “SMOG grading: a new readability formula,” Journal of Reading, vol. 12, pp. 639–646, 1969.
View at: Google Scholar
Readability Formulas, http://www.readabilityformulas.com/.
G. S. Antonarakis and S. Kiliaridis, “Internet-derived information on cleft lip and palate for families with affected children,” The Cleft Palate-Craniofacial Journal, vol. 46, no. 1, pp. 75–80, 2009.
View at: Publisher Site | Google Scholar
C. C. Doak, L. Doak, and J. H. Root, Teaching Patients with Low Literacy Skills, J. B. Lippincott, Philadelphia, Pa, USA, 2nd edition, 1996.
A. Coulter, “Evidence based patient information is important, so there needs to be a national strategy to ensure it,” The British Medical Journal, vol. 317, no. 7153, pp. 225–226, 1998.
View at: Google Scholar
Harris Interactive, “The growing influence and use of health care information obtained online,” 2011, http://www.harrisinteractive.com/vault/HI-Harris-Poll-Cyberchondriacs-2011-09-15.pdf.
View at: Google Scholar
D. L. Greene, A. J. Appel, S. E. Reinert, and M. A. Palumbo, “Lumbar disc herniation: evaluation of information on the internet,” Spine, vol. 30, no. 7, pp. 826–829, 2005.
View at: Publisher Site | Google Scholar
C. Livas, K. Delli, and Y. Ren, “Quality evaluation of the available internet information regarding pain during orthodontic treatment,” Angle Orthodontist, vol. 83, no. 3, pp. 500–506, 2013.
View at: Publisher Site | Google Scholar
A. Gagliardi and A. R. Jadad, “Examination of instruments used to rate quality of health information on the internet: Chronicle of a voyage with an unclear destination,” British Medical Journal, vol. 324, no. 7337, pp. 569–573, 2002.
View at: Publisher Site | Google Scholar
A. Malpani, “Using information therapy to put patients first,” 2012, http://www.healthlibrary.com/book110_chapter1206.htm.
View at: Google Scholar
N. Lawrentschuk, R. Abouassaly, N. Hackett, R. Groll, and N. E. Fleshner, “Health information quality on the internet in urological oncology: a multilingual longitudinal evaluation,” Urology, vol. 74, no. 5, pp. 1058–1063, 2009.
View at: Publisher Site | Google Scholar
A. M. Fast, C. M. Deibert, G. W. Hruby, and K. I. Glassberg, “Evaluating the quality of Internet health resources in pediatric urology,” Journal of Pediatric Urology, vol. 9, no. 2, pp. 151–156, 2013.
View at: Publisher Site | Google Scholar
P. Grewal and S. Alagaratnam, “The quality and readability of colorectal cancer information on the internet,” International Journal of Surgery, vol. 11, no. 5, pp. 410–413, 2013.
View at: Publisher Site | Google Scholar

Copyright

Copyright © 2016 S. Raj et al. This is an open access article distributed under the Creative Commons Attribution License, which permits unrestricted use, distribution, and reproduction in any medium, provided the original work is properly cited.

PDF Download Citation

Download other formats

Order printed copies

Views

4488

Downloads

1590

Citations