Table of Contents Author Guidelines Submit a Manuscript
BioMed Research International
Volume 2019, Article ID 3748091, 12 pages
https://doi.org/10.1155/2019/3748091
Research Article

Top 100 Publications as Measured by Altmetrics in the Field of Central Nervous System Inflammatory Demyelinating Disease

1Department of Neurology, Seoul Medical Center, Seoul, Republic of Korea
2Department of Neurology, Kangdong Sacred Heart Hospital, Hallym University College of Medicine, Seoul, Republic of Korea
3Department of Neurology, Haeundae Paik Hospital, Inje University College of Medicine, Busan, Republic of Korea
4Department of Radiology, Kangdong Sacred Heart Hospital, Hallym University College of Medicine, Seoul, Republic of Korea

Correspondence should be addressed to Jong Seok Bae; moc.revan@eabswl

Received 15 May 2019; Revised 18 September 2019; Accepted 8 November 2019; Published 2 December 2019

Academic Editor: Marco Giannelli

Copyright © 2019 Jee-Eun Kim et al. This is an open access article distributed under the Creative Commons Attribution License, which permits unrestricted use, distribution, and reproduction in any medium, provided the original work is properly cited.

Abstract

Background. Altmetrics analyze the visibility of articles in social media and estimate their impact on the general population. We performed an altmetric analysis of articles on central nervous system inflammatory demyelinating disease (CIDD) and investigated its correlation with citation analysis. Methods. Articles in the 91 journals comprising the “clinical neurology,” “neuroscience,” and “medicine, general, and internal” Web of Science categories were searched for their relevance to the CIDD topic. The Altmetric Explorer database was used to determine the Altmetric.com Attention Score (AAS) values of the selected articles. The papers with the top 100 AAS values were characterized. Results. Articles most frequently mentioned online were primarily published after 2014 and were published in journals with high impact factors. All articles except one were dealt with the issue of multiple sclerosis. Most were original articles, but editorials were also common. Novel treatments and risk factors are the most frequent topics. The AAS was weakly correlated with journal impact factors; however, no link was found between the AAS and the number of citations. Conclusions. We present the top 100 most frequently mentioned CIDD articles in online media using an altmetric approach. Altmetrics can rapidly offer alternative information on the impact of research based on a broader audience and can complement traditional metrics.

1. Introduction

Traditionally, the impact of each scientific article is indirectly measured by its citation count or the impact factor of the journal publishing it. The impact factor is calculated as the average citation number of the articles that were published in that journal during the preceding two years (number of total citations/number of total citable articles). However, accessibility to websites is dramatically increasing worldwide, and the number of social media users has reached almost 2.46 billion in 2017 [1]. One-third of people worldwide are anticipated to use social media in 2021 [1]. Because of these shifts, the channels and audiences for science communication are inexorably increasing. Altmetrics, also called alternative metrics, capture the online visibility of scholarly material and indirectly present its influence on social networks [2]. Altmetrics quantify the frequency of mentions in online channels including news outlets, science blogs, Twitter, Facebook, Sina Weibo, Wikipedia, public policy documents, peer review platforms, research highlights on Faculty of 1000, reference manager such as Mendeley, and multimedia (YouTube and Q&A) [3]. The Altmetric.com Attention Score (AAS), one of the commonly used altmetric tools, is calculated depending on the intensity, types (e.g., retweets are scored lower than original tweets), and sources of attention (e.g., articles shared through news outlets are weighted 8, but those shared on Twitter are weighted 1) [3]. A high AAS, as calculated by this method, represents noteworthy interest in an article within social media outlets [4]. Altmetrics have certain advantages over traditional metrics: the former can measure the effect of scientific works outside the academic community and can estimate the impact of discoveries more quickly than citation analysis [4].

Multiple sclerosis (MS), the most representative form of central nervous system inflammatory demyelinating disease (CIDD), has been an important issue in the health care community because of its relatively high prevalence, chronicity, and massive social burden. MS typically develops in young adults aged 20–50 years, and its incidence peaks in the fourth decade of life [5]. These young adult patients are especially familiar with Internet use, and they actively acquire and spread information on the disease by social media or other digital outlets [6]. There are numerous official and unofficial social networks, web pages, and blogs on the topic of MS [7].

Here, our purpose is to identify the most influential articles in the field of CIDD in online media and to investigate the characteristics of those articles. We hypothesized that traditional metrics and altmetrics were not necessarily related. We compared the ASS to a traditional citation analysis to find the relationship between the two metrics.

2. Materials and Methods

2.1. Article Selection and Analysis

We searched journals in the categories of “clinical neurology,” “neuroscience,” and “medicine, general, and internal” in the 2016 edition of Web of Science (Thomson Reuters, New York, NY) to further identify the CIDD-related articles that were most commonly mentioned in digital media. A total of 91 journals were selected, and articles published in those journals were subjected to further analysis. We used the AAS as a weighted measure of the attention received by each article in digital media [3]. The AAS of articles in selected journals were searched separately in the Altmetric Explorer database (Altmetric LLP, London, UK) on May 25, 2018. All articles were compiled as a single database and rearranged in order of descending AAS. To retrieve CIDD-related articles, we chose articles with the following terms in their titles: “multiple sclerosis,” “clinically isolated syndrome,” “neuromyelitis optica,” “neuromyelitis optica spectrum disorder (NMOSD),” “optic neuritis,” “myelitis,” “demyelinating disease,” “Devic’s disease,” “aquaporin-4 antibody,” “neuromyelitis optica-immunoglobulin G,” “Schilder’s disease,” “Schilder’s diffuse sclerosis,” “diffuse myelinoclastic sclerosis,” “Balo concentric sclerosis,” “Marburg multiple sclerosis,” “solitary sclerosis,” “acute disseminated encephalomyelitis,” or “acute hemorrhagic leukoencephalitis.” After reviewing the original texts to confirm their relevance to CIDD, we identified the top 100 articles by AAS. To avoid the risk of overestimating public attention because of single articles dealing with many different diseases, we excluded articles that dealt with multiple diseases (e.g., an article that evaluated the effect of living near a major road on the risk of multiple sclerosis, dementia, and Parkinson’s disease had the highest AAS in the present study, but it was excluded from further evaluation) [8]. Articles were reviewed for the selection process regardless of language, scholarly identifier, or document type.

These mixed methods study utilized quantitative analyses, qualitative assessment, and an extensive review of literature. The following information was extracted from the top 100 most highly mentioned CIDD-related articles in online media: article title, year of publication (if electronic publishing was opened ahead of formal publication, the year of electronic publishing was selected for further analysis), published journal, journal impact factor, country of origin, type of article, main topics, subject of article (e.g., MS, NMOSD, and myelitis), AAS, and number of citations. Country of origin was defined by the affiliation of the first author. If the first author had more than one affiliation, then the affiliation of the corresponding author was used to determine the country of origin. The journal impact factor was extracted from the 2017 edition of Web of Science. The traditional citation count for each article was obtained from Scopus and compared to the AAS.

The study was exempted from approval by an ethics committee because it was a bibliometric analysis.

2.2. Statistical Analysis

All statics were analyzed using SPSS Statistics 20 (SPSS, Chicago, IL, USA). Pearson’s correlation analysis was performed to assess the relationship between AAS and journal impact factor (or citation count). was regarded as statistically significant.

3. Results

We identified 100 articles with the highest altmetric scores (Table 1). AAS values were in the range of 113 to 1302 (median 214.5). The median journal impact factor of the articles with the top 100 altmetric scores was 7.690 (range 2.45–79.26). The median number of citations per article was 21 (range 0–765). Altmetric.com has tracked attention paid to articles mentioned in each source since 2011. The identified articles were published between 1990 and 2018, and 85% of the articles were published after 2014 (Figure 1). Articles on the top 100 list originated from 15 different countries (Table 2). Approximately 60% of the top 100 articles originated in Northern America. The remaining high-ranked countries of origin were on the European continent. Interestingly, the top 100 articles discussed online were published mostly in journals with high impact factors (Table 3). Among 23 journals, Neurology (n=31) has the most articles ranked in the top 100, followed in order, by the Journal of the American Medical Association Neurology (n=14), The Lancet (n=11), the Journal of Neurology, Neurosurgery and Psychiatry (n=10), and The New England Journal of Medicine (n=8). Except for one article explaining clinical biomarkers to differentiate inflammatory myelopathy from other causes, the subject of all the articles was MS. No articles on the subject of NMOSD were in the top 100. However, two NMOSD articles had notably high AAS values. One article suggesting 2015 NMOSD diagnostic criteria ranked 112 (AAS = 98), and an article regarding the clinical course and therapeutic efficiency of NMOSD ranked 125 (AAS = 88) [9, 10].

Table 1: List of the top 100 articles by Altmetric.com Attention Scores.
Figure 1: Articles with the top 100 highest Altmetric.com Attention Scores, sorted by the year of publication.
Table 2: Countries of origin of articles highly mentioned online.
Table 3: Journals where the central nervous system inflammatory demyelinating disease articles with the highest Altmetric.com Attention Scores were published.

Original articles were the most frequent type among the top 100 highly mentioned CIDD papers, and 60% of the original articles were clinical observational studies (Table 4). Approximately 10% of highly mentioned articles were reviews, meta-analyses, or articles suggesting practical guidelines or diagnostic criteria. Editorial articles were also included in the list of high-ranking publications (Table 4). Most of these editorial articles summarized notable recent studies and provided expert opinions regarding new treatments (including B-cell depletion treatments such as ocrelizumab and alemtuzumab, BCG vaccines, hematopoietic stem-cell transplantation, and percutaneous transluminal venous angioplasty) and risk factors (such as coffee consumption, chronic cerebrospinal venous insufficiency, diet, vitamin D, and smoking). The main issues addressed by the top 100 articles were mostly related to disease treatment, and many of them were comparative clinical trials (Table 5). The specific treatments receiving the most attention are presented in Table 6. One article was a retrospective cohort study presenting the complication of malignancy after mitoxantrone treatment in MS. The second most frequent class of topics consisted of the risk factors for MS development and aggravation (Table 6). Notably, several high-ranking articles concerned quality-of-life or economic issues.

Table 4: Types of articles with top 100 Altmetric.com Attention Scores.
Table 5: General issues discussed in the highly mentioned articles on central nervous system inflammatory demyelinating disease.
Table 6: Topics on treatment and risk factor of central nervous system inflammatory demyelinating disease that achieved high attention in social outlets.

Regarding the correlation between AAS and journal impact factor among the top 100 articles, a weak positive correlation was revealed (r = 0.2474; ) (Figure 2). However, there was no significant correlation between AAS and citation counts.

Figure 2: Correlation between Altmetric.com Attention Scores and journal impact factors among the top 100 articles on central nervous system inflammatory demyelinating disease mentioned most online.

4. Discussion

We performed an altmetric analysis of CIDD articles to examine their influences on not only academia but also the general public. We thus obtained a list of the 100 articles in the field of CIDD that were most frequently mentioned in web media according to weighted AAS. These results show trends in scientific research and social attention on a shorter time lag than traditional citation analysis. Among our top 100 articles, AAS showed correlation with journal impact factor but not with citation count, which means that altmetrics are correlated with traditional metrics to some extent but do not convey identical information.

Traditional citation analysis has a long history as a tool for measuring the impact of an article, authors, and contributed institutes/countries. Citation analysis offers a “best-seller list” of the specific diseases that receive the most attention in academia. However, this form of analysis has several limitations. In practice, citations of an article start 1–2 years after its initial publication and peak between 3 and 10 years [11]. After considerable periods, some important articles cease to be frequently cited because their substance has been absorbed into the current body of common knowledge, the phenomenon called “obliteration by incorporation” [12]. For this reason, traditional analysis best captures the actual impact of a paper 10 to 20 years after publication [13, 14]. Altmetrics are a new class of metrics that can indicate the impact of an article in a more rapid and responsive way than citation analysis. Previous results comparing altmetrics and citation analysis have shown that new web-based metrics are especially sensitive to the latest news [15, 16]. Reflecting this characteristic, most of the top 100 CIDD articles in our study were published after 2014, and a plurality of articles were published in 2016. Furthermore, when we considered the relationship between citation number and AAS, we could not find any correlation. A substantial number of articles on the top 100 list were cited fewer than 10 times, which might be due to the scope of the audience, the document types, and the time since publication. Our results support the idea that citation analysis and altmetrics do not measure the same construct; they are complementary rather than substitutable. The absence of a strong correlation between new metrics and citation analysis was consistently observed in other studies [1723]. Only Mendeley, an online reference manager, showed a moderate positive link with citation analysis [19, 20].

In our correlation analysis, AAS showed a weak positive correlation with a journal impact factor. In addition, the top 5 most represented journals, which included 74% of the listed articles, have impact factors of more than 7. This result may have occurred because articles published in higher impact journals are generally more interesting in their own right or because of the fame and wide reach of these journals. Several factors are known to influence AAS, such as journal impact factor, article length, number of collaborators or references, presence of a press release, free accessibility, document type, and funding [16, 24]. Several factors, such as reference count and collaborative practices, positively affect both citations and altmetrics [16]. However, the impacts of other factors are not identical [16]. Longer papers typically receive more citations, but the opposite trend can be seen on social media platforms [16]. Editorial and news documents, which are rarely cited, are frequently mentioned document types on Twitter [16]. Simple declarative titles that present key conclusions are associated with high AAS values [2527]. Our study also showed substantial inclusion of editorial articles, which distill topics to their essentials, and are easier to understand than research papers. Brevity and understandability are important factors in the dissemination of articles on websites [25].

The most frequently mentioned topics were associated with novel therapeutics and risk factors (especially correctable environmental factors). Articles concerning disease mechanisms, clinical characteristics, and diagnosis also accounted for a portion of the list. However, compared with traditional citation analysis, altmetrics show increased attention to treatment and decreased attention to diagnosis and disease mechanisms [28]. Similarly, articles that address socioeconomic and quality-of-life issues of CIDD patients are also highly ranked in terms of altmetrics, an outcome that is seldom observed in citation analysis [28]. Other researchers also reported that research papers relating to humanities and social science are most frequently found on online media, in contrast to the pattern in conventional media [16]. Our findings indirectly represent the difference between audience and user groups (academic-based versus a more inclusive public audience) and their interest in conventional and social media. The top 100 articles mostly originated in North America and Europe. These results may have occurred because many core studies were performed in those areas. Another influence might be the large volume of attention paid to MS in Northern America and Europe because of its relatively high prevalence and because of widespread Internet use [1, 5].

Several aspects of this study should be considered. First, altmetrics are new methods of bibliometrics, and their value was uncertain until now. Several inherent limitations of altmetrics have been discussed recently [4, 21, 29]. Social media are at risk of overestimating the impact of research with sensational outcomes [21, 29]. Articles with negative findings or attractive headlines are likely to have high AAS values [21]. Some scientists have reported that metrics from online platforms do not accurately reflect the quality research [30]. AAS scores cannot distinguish the depth of online mentions although the algorithm is adjusted to weight scores by the source and type of attention [22]. Furthermore, due to the anonymity of social media, the demographics of the users referencing the articles (scientists or the lay public) and the validity of the data are often not certain [21, 29]. As in traditional citation analysis, altmetrics may be influenced by temporal bias [12]. Second, we used only altmetric data offered by Altmetric.com in this study. Social media metrics in each data aggregator can differ due to methodological (e.g., selections on the data collection) and reporting choices (e.g., data aggregation approaches) and have both advantages and disadvantages [31]. Other tools such as ALS-PLoS, ImpactStory, and Plum Analytics can also be used to track altmetrics data. Third, we did not perform subgroup analyses based on the attention source subtype. Further studies focusing on difference in the social media subset are expected.

5. Conclusion

We identified the 100 research outputs most discussed on online platforms shared by a broad audience of scientists and laypeople. There was some discordance between traditional citation analysis and altmetrics analysis of CIDD articles. Data from altmetrics may not accurately represent scientific quality or importance, but they can reflect the dissemination of research through the general population, including the scientific community, on the web. The influence of online media will inevitably increase in the future. Altmetrics can provide a broad, real-time web-based reflection of research and can complement traditional analysis as a tool to evaluate the impact of studies. Further research will be needed to assess the validity of altmetrics and explore the factors that influence them.

Abbreviations

AAS:Altmetric.com Attention Score
CIDD:Central nervous system inflammatory demyelinating disease
CSF:Cerebrospinal fluid
IF:Impact factor
MS:Multiple sclerosis
NMOSD:Neuromyelitis optica spectrum disorder
No:Number.

Data Availability

The data used to support the findings of this study are available from the corresponding author upon request.

Conflicts of Interest

The authors declare that there are no conflicts of interest.

References

  1. The Statistics Portal, Social Media Statistics & Facts, 2018, , 2018, http://www.statista.com/topics/1164/social-networks/.
  2. J. Chavda and A. Patel, “Measuring research impact: bibliometrics, social media, altmetrics, and the BJGP,” British Journal of General Practice, vol. 66, no. 642, pp. e59–e61, 2016. View at Publisher · View at Google Scholar · View at Scopus
  3. Altmetric Support, “How is the altmetric attention score calculated?” 2018, https://help.altmetric.com/support/solutions/articles/6000060969-how-is-the-altmetric-attention-score-calculated. View at Google Scholar
  4. N. S. Trueger, B. Thoma, C. H. Hsu, D. Sullivan, L. Peters, and M. Lin, “The altmetric score: a new measure for article-level dissemination and impact,” Annals of Emergency Medicine, vol. 66, no. 5, pp. 549–553, 2015. View at Publisher · View at Google Scholar · View at Scopus
  5. R. Milo and E. Kahana, “Multiple sclerosis: geoepidemiology, genetics and the environment,” Autoimmunity Reviews, vol. 9, no. 5, pp. A387–A394, 2010. View at Publisher · View at Google Scholar · View at Scopus
  6. A. Smith, M. Anderson, and Pew Research Center, Social Media Use in 2018, 2018, , 2018, https://www.pewinternet.org/2018/03/01/social-media-use-in-2018/.
  7. L. Lavorgna, F. Brigo, M. Moccia et al., “e-Health and multiple sclerosis: an update,” Multiple Sclerosis Journal, vol. 24, no. 13, pp. 1657–1664, 2018. View at Publisher · View at Google Scholar · View at Scopus
  8. H. Chen, J. C. Kwong, R. Copes et al., “Living near major roads and the incidence of dementia, Parkinson’s disease, and multiple sclerosis: a population-based cohort study,” The Lancet, vol. 389, no. 10070, pp. 718–726, 2017. View at Publisher · View at Google Scholar · View at Scopus
  9. D. M. Wingerchuk, B. Banwell, J. L. Bennett et al., “International consensus diagnostic criteria for neuromyelitis optica spectrum disorders,” Neurology, vol. 85, no. 2, pp. 177–189, 2015. View at Publisher · View at Google Scholar · View at Scopus
  10. I. Kleiter, A. Gahlen, N. Borisow et al., “Neuromyelitis optica: evaluation of 871 attacks and 1,153 treatment courses,” Annals of Neurology, vol. 79, pp. 206–216, 2016. View at Publisher · View at Google Scholar · View at Scopus
  11. W. Marx, H. Schier, and M. Wanitschek, “Citation analysis using online database: feasibilities and shortcomings,” Scientometrics, vol. 52, pp. 59–82, 2001. View at Google Scholar
  12. E. Garfield, “100 citation classics from the journal of the American medical association,” JAMA, vol. 257, pp. 52–59, 1987. View at Google Scholar
  13. D. M. Albert, “Analysis of the Archives’ most frequently cited articles,” Archives of Ophthalmology, vol. 106, pp. 465–470, 1988. View at Publisher · View at Google Scholar
  14. G. M. Hall, “BJA citation classics 1945–1992,” British Journal of Anaesthesia, vol. 80, pp. 4–6, 1998. View at Publisher · View at Google Scholar · View at Scopus
  15. R. Araújo, A. A. Sorensen, S. Konkiel, and B. R. Bloem, “Top altmetric scores in the Parkinson’s disease literature,” Journal of Parkinson’s Disease, vol. 7, pp. 81–87, 2017. View at Publisher · View at Google Scholar · View at Scopus
  16. S. Haustein, R. Costas, and V. Larivière, “Characterizing social media metrics of scholarly papers: the effect of document properties and collaboration patterns,” PLoS One, vol. 10, Article ID e0120495, 2015. View at Publisher · View at Google Scholar · View at Scopus
  17. M. Thelwall, S. Haustein, V. Larivière, and C. R. Sugimoto, “Do Altmetrics work? Twitter and ten other social web services,” PLoS One, vol. 8, no. 5, Article ID e64841, 2013. View at Publisher · View at Google Scholar · View at Scopus
  18. D. Barbic, M. Tubman, H. Lam, and S. Barbic, “An analysis of altmetrics in emergency medicine,” Academic Emergency Medicine, vol. 23, no. 3, pp. 251–268, 2016. View at Publisher · View at Google Scholar · View at Scopus
  19. S. Haustein, V. Larivière, M. Thelwall et al., “Tweets vs. Mendeley readers: how do these two social media metrics differ?” Information Technology, vol. 56, pp. 207–215, 2014. View at Publisher · View at Google Scholar
  20. A. Amath, K. Ambacher, J. J. Leddy, T. J. Wood, and C. J. Ramnanan, “Comparing alternative and traditional dissemination metrics in medical education,” Medical Education, vol. 51, no. 9, pp. 935–941, 2017. View at Publisher · View at Google Scholar · View at Scopus
  21. E. M. O’Connor, G. J. Nason, F. O’Kelly, R. P. Manecksha, and S. Loeb, “Newsworthiness vs scientific impact: are the most highly cited urology papers the most widely disseminated in the media?” BJU International, vol. 120, no. 3, pp. 441–454, 2017. View at Publisher · View at Google Scholar · View at Scopus
  22. K. Delli, C. Livas, F. K. L. Spijkervet, and A. Vissink, “Measuring the social impact of dental research: an insight into the most influential articles on the web,” Oral Diseases, vol. 23, no. 8, pp. 1155–1161, 2017. View at Publisher · View at Google Scholar · View at Scopus
  23. A. F. Barakat, N. Nimri, M. Shokr et al., “Correlation of altmetric attention score and citations for high-impact general medicine journals: a cross-sectional study,” Journal of General Internal Medicine, vol. 34, no. 6, pp. 825–827, 2019. View at Publisher · View at Google Scholar · View at Scopus
  24. R. Haneef, P. Ravaud, G. Baron et al., “Factors associated with online media attention to research: a cohort study of articles evaluating cancer treatments,” Research Integrity and Peer Review, vol. 2, no. 1, p. 9, 2017. View at Publisher · View at Google Scholar
  25. N. Di Girolamo and R. M. Reynders, “Health care articles with simple and declarative titles were more likely to be in the altmetric top 100,” Journal of Clinical Epidemiology, vol. 85, pp. 32–36, 2017. View at Publisher · View at Google Scholar · View at Scopus
  26. R. Costas, Z. Zahedi, and P. Wouters, “Do “Altmetrics” correlate with citations? extensive comparison of altmetric indicators with citations from a multidisciplinary perspective,” Journal of the Association for Information Science and Technology, vol. 66, no. 10, pp. 2003–2019, 2015. View at Publisher · View at Google Scholar · View at Scopus
  27. I. Peters, P. Kraker, E. Lex, C. Gumpenberger, and J. Gorraiz, “Research data explored: an extended analysis of citations and altmetrics,” Scientometrics, vol. 107, no. 2, pp. 723–744, 2016. View at Publisher · View at Google Scholar · View at Scopus
  28. J.-E. Kim, K. M. Park, Y. Kim, D. Y. Yoon, and J. S. Bae, “Citation classics in central nervous system inflammatory demyelinating disease,” Brain and Behavior, vol. 7, no. 6, Article ID e00700, 2017. View at Publisher · View at Google Scholar · View at Scopus
  29. H. R. Warren, N. Raison, and P. Dasgupta, “The rise of altmetrics,” JAMA, vol. 317, no. 2, pp. 131-132, 2017. View at Publisher · View at Google Scholar · View at Scopus
  30. L. Bornmann and R. Haunschild, “Do altmetrics correlate with the quality of papers? a large-scale empirical study based on F1000Prime data,” PLoS One, vol. 13, no. 5, Article ID e0197133, 2018. View at Publisher · View at Google Scholar · View at Scopus
  31. Z. Zahedi and R. Costas, “General discussion of data quality challenges in social media metrics: extensive comparison of four major altmetric data aggregators,” PLoS One, vol. 13, no. 5, Article ID e0197326, 2018. View at Publisher · View at Google Scholar · View at Scopus