CC BY
23Central Asian Journal of Medical Hypotheses and Ethics
2021; Vol 2 (2)
© 2021 by the authors. This work is licensed under Creative Commons Attribution 4.0 International License https://creativecommons.org/licenses/by/4.0/
eISSN: 2708-9800 https://doi.org/10.47316/cajmhe.2021.2.2.01
RESEARCH METHODOLOGY
REVIEW
ALTMETRICS AND CITATION METRICS AS COMPLEMENTARY INDICATORS FOR
RESEARCH MANAGEMENT
Received: April 15, 2021 Accepted: May 18, 2021
Burhan Fatih Koçyigit1 http://orcid.org/0000-0002-6065-8002 Ahmet Akyol2 http://orcid.org/0000-0002-8953-5196
department of Physical Medicine and Rehabilitation, Faculty of Medicine, Kahramanmara§ Sûtçû ¡mam University, Kahramanmara§, Turkey
2Physiotherapy and Rehabilitation Application and Research Center, Hasan Kalyoncu University, Gaziantep, Turkey
Corresponding author: Burhan Fatih Koçyigit, MD, Department of Physical Medicine and Rehabilitation, Faculty of Medicine, Kahramanmara§ Sûtçû imam University, Kahramanmara§, Turkey; Twitter handle: @BurhanFatihKoy1; E-mail: [email protected]
Abstract
The quality and impact of scholarly articles are traditionally evaluated using metrics that put citation counts at the center. In the era of digitization, advanced online platforms and social media have transformed the dissemination of scientific information results and introduced new metrics for evaluating the influence of scholarly outputs and planning research studies. Alternative metrics (altmetrics) have emerged as tools for immediate measuring of scholarly outputs upon their online publication and dissemination through numerous online platforms, including social media channels. The Altmetrtic Attention Score by Altmetric.com has gained its global importance as an integral indicator for evaluation of constantly changing societal influence of individual articles. To a large extent, this and other social media metrics complement established citation metrics, allowing to visualize and predict implications at early stages of post-publication promotion. This article overviews characterstics of various altmetric tools and their growing roles for planning and evaluating scientific research.
Keywords: Research, Bibliographies as topic, Social media, Altmetrics, Altmetric attention score, Twitter, Mendeley How to cite: Kogyigit BF, Akyol A. Altmetrics and citation metrics as complementary indicators for research management. Cent Asian J Med Hypotheses Ethics 2021:2(2):79-84. https://doi.org/10.47316/cajmhe.2021.2.2.01
INTRODUCTION
The modern-day scholarly publishing activities priortize strategies for analzying large volumes of scientific litearure and visualizing trends and directions in research. It is becoming increasingly important to weigh the global influence of articles while bookmarking and archiving in individual researcher libraries.
Pre- and post-publication peer reviews are currently the main quality evaluation mechanisms that influence the overall post-publication promotion and citation aggregation [1]. The established research evaluation strategies are largely based on citations, which are
viewed as 'hard currencies' of individual researchers and related institutions. Citation-based evaluations are compounded by lengthy time periods which are required to disseminate publications and incorporate related references in traditional hubs of knowledge accumulation - subsequent scholarly articles. In other words, published articles become references and influence bibliographies over a relatively long period.
Over the past decade, advances in online dissemination of information through various public channels and scholarly networks have substantially accelerated data sharing and researcher engagement in the global post-
publication communication [2]. As a result, new metrics have emerged to quantify 'societal influence' of scholarly publications and supply the publishing enterprise with new dimensions for research evaluation. The term alternative metrics (altmetrics) has been proposed to complement established citation metrics and to diversify research evaluation [3].
Altmetric analyses allow mapping societal attention and distinguishing popular online channels for information dissemination [4, 5]. The weight (prestige) and popularity of such online channels differ across countries [6].
The Altmetric.com platform tracks records and feeds from diverse social media platforms, blogs, news outlets, scholarly networking and bookmarking sites, public policy documents, and other online sources, all of which are deemed important for an alternative view on scholarly reuse and influence on post-publication communication [7, 8]. Altmetric.com owns a proprietary algorithm for quantifiying an integral score based on an assumption that online channels have differing powers to reflect public attention to publicized articles [9]. Compared with citation metrics, the main advantage of altmetrics is its immediacy effect and real-life monitoring of public attention to scholarly articles. The immediacy effect is due to the ease of public attention aggregation, reflecting almost all changes in the use and dissemination of scientific information by scholars and other public representatives.
The aim of this article is to overview popular altmetric tools and distinguish their main differences from established research evaluation methods. The results of this overview may contribute to increasing the awareness of altmetric tools and their uses for researchers and editors.
ALTMETRIC ATTENTION SCORE
To visualize the Altmetric Attention Score (AAS) of individual articles, 'altmetric it tool can be downloaded from https://www.altmetric.com/products/free-
tools/bookmarklet/. This tool allows free monitoring of AAS. The altmetric multicolor donut displays all contributing social-media components of AAS in its ring and an aggregate score at the center. The AAS is an indicator of public attention to an article across various online platforms [10]. Although the platforms contributing to the AAS calculation are publicly listed, and this list is continuously expanding
(https://www.altmetric.com/about-our-data/our-sources/; Box 1), the proprietary calculation algorithm is unavailable to the public [11].
Box 1. Altmeric Attention Score Sources
Sources
• Public policy documents
• Online reference managers
• Post-publication peer review platforms
• News
• Wikipedia
• Social media (Facebook, Twitter, LinkedIn,
Google+, Sin Weibo and Pinterest)
• Multimedia and other online platforms
(YouTube, Reddit, Q&A)
• Citations
• Blogs
• Patents
• Post-publication peer review platforms
(Pubpeer and Publons)
• Research highlights
Each color in the altmetric donut indicates attention from a separate platform: light blue refers to Twitter, red to news outlets, yellow to blogs, etc. (https://www.altmetric.com/about-our-data/the-donut-and-score/). The 'weight' of each color in the altmetric donut varies depending on attention quantitative values. The AAS is calculated based on three factors: volume, sources, and authors (https://www.altmetric.com/about-our-data/the-donut-and-score/). The AAS increases as more people mention it; only 1 attention point from a single person on a platform is acceptable. For example, if a user tweets about the same article more than once, Altmetric.com counts the initial activity only. Attention arising from different platforms variably contributes to the AAS. Altmetric.com takes into account whether there is a bias against a particular journal or publisher. It evaluates how often the referring user mentions the promoted articles.
PlumX METRICS
PlumX is an online tool that helps to obtain information on the use and influence of various scientific products. PlumX obtains data from web sources. PlumX processes alternative parameters such as download counts, abstract views, comments on online platforms, social-media activities, as well as citations. It provides an insight into the public discussion of scholarly and other items [12, 13]. Scientific products in PlumX are named as artifacts. All data obtained by PlumX are processed and presented as "Plum Print", an infographic with separate circles for five categories (usage, captures, mentions, social media, and citations) [14]. The 'usage'
category covers article-level statistics such as abstract views, text downloads, clicks, figure and PDF views, and video acts. The "captures" category includes bookmarkings, adding to favourites lists, and exports. The 'mentions' category covers evaluations in Amazon, comments in Facebook and Slideshare, Forum subjects, Wikipedia links, and mentions in blogs. The 'social media' category takes into account likes, shares and tweets associated with artifacts. The 'citations' category analyzes artifacts using traditional metrics (https://plumanalytics.com/learn/about-metrics/). Plum Print provides an intergrated view of alternative and traditional metrics. The visualization is available in green for 'usage', purple for 'captures' category, yellow for 'mentions', light blue for 'social media', and orange for 'citations'. In addition to artifact-based data, PlumX also generates researcher and institute-level data. PlumX dashboard helps to customize graphs while generating institutional and researcher metrics.
Twitter has become a globally popular communication platform. Numerous special interest groups create their Twitter accounts for scholarly purposes. Twitter account holders discuss their scientific ideas, disseminate research results, and expand networks with professionals [15]. Twitter is well equipped for commenting on and disseminating research ouputs [16].
Journals and editors actively use Twitter. Their aim is to keep scholars abreast of latest developments in their field of interest [17]. Twitter-sourced altmetrics reflect public's interest surrounding scholarly articles and other products. Non-professional tweets are less likely to predict citations than academic tweets [18]. Although Twitter provides useful altmetrics for evaluating scientific communications, there is uncertainty over the accuracy of the content. The inability of professionals to actively use Twitter owing to the high intensity of their workloads may limit the spread of scientific information. In addition, bots and trolls can indiscriminately generate tweets, some of which contain misinformation and distorted altmetric analyses [16, 19, 20].
MENDELEY BOOKMARKINGS
Mendeley is one of the most frequently used online reference managers that allows to combine bookmarking with reference management on a single platform [21]. It also offers scholarly networking and collaboration. Article PDFs are exported to the Mendeley interface, with metadata being automatically exposed. The Mendeley reader counts reflect article bookmarkings by online users [22]. The reader counts varies according to the frequency of adding articles to individual libraries.
Researchers who add articles to their Mendeley libraries are likely to cite these items in their future research outputs [23].
COMPLEMENTING ALTMETRICS WITH CITATION METRICS
Citations are taken into account for calculation of traditional impact metrics. In the digitized world, not just citations, but all other post-publication promotion and other activities can be monitored and analyzed. The resultant metrics reflect attention surrounding articles, not necessarily their quality.
Altmetrics and citation metrics are variably associated across disciplines [24, 25]. However, altmetrics provide early post-publication information, well in advance of citation aggregation. Altmetrics should not be considered as a substitute for traditional citations. These should be viewed as complementary parameters.
SAN FRANCISCO DECLARATION ON RESEARCH ASSESSMENT
San Francisco Declaration on Research Assessment (DORA; https://sfdora.org/read/ ) was issued by scientists and editors at the American Cell Biology Association meeting in San Francisco in December 2012. The journal editors voiced their concerns over the misuses of the Journal Impact Factor that became a proxy of journal and article quality. The DORA highlighted some action points for moving away from the JIF uses as a single metric and switiching to a combination of reliable metrics, including altmetrics. As a result, academics, funding agencies, and scientific institutions have stepped up their work to develop more effective tools and faster and more efficient ways to assess the quality of researcher contributions, rather than focusing on JIF and related metrics [26, 27].
SCHOLARLY IMPLICATIONS OF SOCIAL MEDIA AND ALTMETRICS IN CENTRAL ASIA
Central Asian researchers are mostly optimistic over the uses of Internet platforms and social media for scholarly purposes. However, they are also concerned over the privacy issues and local restrictive regulations [6, 28]. The active scholarly use of online platforms and social media may not only boost altmetrics, but also improve Central Asian users' online presence and influence.
The use of various web-based platforms facilitates access to various sources of scientific information, networks of scholars, and hypothesis-generating ideas [6]. Online communications and networking in Central Asia and beyond may help create a basis for future research studies.
Altmetrics and social media may improve hypothesis generation, research planning, and post-publication analyses. New directions of research can be planned in view of altmetric analyses and ranking trending articles.
Social media platforms display online comments, including positive and negative remarks which may help generate testable hypotheses and plan new studies. Identifying areas of great importance for social media users may also prompt funding agencies to relocate their grants to certain research projects.
It should also be noted that social media and altmetrics are confounded by some inherent negative aspects such as data manipulation. In fact, number of views, downloads, and mentions can be increased using trolls, mechanistic (robotics) and bot accounts. To correct for all these negative confounders, altmetric analyses should be more accurate and detailed. At times, analysing separate components of the AAS brings more valuable information than blindly relying on the integral value of this score. Social media metrics should always be accompanied with citations and expert evaluations [16].
CONCLUSION
Altmetric tools have emerged to advance scientific communications, networking, and research planning. With unprecented popularity of various social media platforms, altmetric analyses have become more objective reflections of the societal impact of scholarly outputs. Unsurprisngly, more and more altmetric tools are being employed by journals, publishers, and
REFERENCES
databases to visualize new dimensions of impact measurement.
Altmetrics are complementary to established bibliometrics such as citation counts and JIF. Their combination is believed to be a better reflection of research productivity and impact. As with bibliometrics, altmetric analyses should be corrected for time periods and patterns of certain disciplines and countries. Limitations and manipulation risks should be also taken into account. Therefore, any quantitative bibliometric and altmetric analyses should be accompanied with context analyses and expert evaluations.
FUNDING
None
AUTHOR CONTRIBUTIONS
Both authors substantively contributed to the drafting of the initial and revised versions of this article. They take full responsibility for the integrity of all aspects of the work.
CONFLICTS OF INTEREST
Both authors have completed the ICMJE Disclosure Form (http://www.icmje.org/disclosure-of-interest/; available on request from the corresponding author). Both authors declare that there are no potential conflicts of interest.
DISCLAIMER
No part of this article is copied or published elsewhere in whole or in part.
1. Scott A. Peer review and the relevance of science. Futures 2007;39(7):827-845.
2. Zimba O, Gasparyan AY. Social media platforms: a primer for researchers. Reumatologia 2021 ;59(2):68-72.
3. Smith DR, Watson R. Career development tips for today's nursing academic: bibliometrics, altmetrics and social media. J Adv Nurs 2016;72(11):2654-2661.
4. Mullins CH, Boyd CJ, Corey BL. Examining the correlation between altmetric score and citations in the general surgery literature. J Surg Res 2020;248:159-164.
5. Konkiel S. Altmetrics: diversifying the understanding of influential scholarship. Palgrave Communications 2016;2(1):1-7.
6. Gaur PS, Gupta L. Social media for scholarly communication in Central Asia and its Neighbouring countries. J Korean Med Sci 2021;36(4):e36.
7. Ortega JL. Reliability and accuracy of altmetric providers: a comparison among Altmetric. Com, PlumX and Crossref Event Data. Scientometrics 2018;116(3):2123-2138.
8. Carroll C, Tattersall A. Research and policy impact of trials published by the UK national institute of health research (2006-2015). Value Health 2020;23(6):727-733.
9. Galligan F, Dyas-Correia S. Altmetrics: rethinking the way we measure. Serials review 2013;39(1):56-61.
10. Elmore SA. The altmetric attention score: what does it mean and why should I care? Toxicol Pathol 2018;46(3):252-255.
11. Trueger NS, Thoma B, Hsu CH, Sullivan D, Peters L, Lin M. The altmetric score: a new measure for article-level dissemination and impact. Ann Emerg Med 2015;66(5):549-553.
12. Lindsay JM. PlumX from plum analytics: not just altmetrics. J Electron Resour Med Libr 2016;13(1):8-17.
13. Tornberg HN, Moezinia C, Wei C, Bernstein Sa, Wei C, Al-Beyati R, et al. Assessing the dissemination of COVID-19 articles across social media with altmetric and PlumX metrics: correlational study. J Med Internet Res 2021 ;23(1):e21408.
14. Torres-Salinas D, Gumpenberger C, Gorraiz J. PlumX as a potential tool to assess the macroscopic multidimensional impact of books. Front Res Metr 2017;2:5.
15. Ke Q, Ahn YY, Sugimoto CR. A systematic identification and analysis of scientists on Twitter. PloS One 2017;12(4):e0175368.
16. Gasparyan AY, Yessirkepov M, Voronov AA, Maksaev AA, Kitas GD. Article-level metrics. J Korean Med Sci 2021 ;36(11 ):e74.
17. Zimba O, Radchenko O, Strilchuk L. Social media for research, education and practice in rheumatology. Rheumatol Int 2020;40(2):183-190.
18. Yu H. Context of altmetrics data matters: An investigation of count type and user category. Scientometrics 2017;111(1 ):267-283.
19. Jamison AM, Broniatowski DA, Quinn SC. Malicious actors on twitter: a guide for public health researchers. Am J Public Health 2019;109(5):688-692.
20. Ahmed S, Gupta L. Social media for medical journals. Cent Asian J Med Hypotheses Ethics 2020;1(1):26-32.
21. Bornmann L. Alternative metrics in scientometrics: a meta-analysis of research into three altmetrics. Scientometrics 2015;103:1123-1144.
22. Meschede C, Siebenlist T. Cross-metric compatability and inconsistencies of altmetrics. Scientometrics 2018;115(1 ):283-297.
23. Pooladian A, Borrego A. A longitudinal study of the bookmarking of library and information science literature in Mendeley. J Informetrics 2016;10(4):1135-1142.
24. Livas C, Delli K. Looking beyond traditional metrics in orthodontics: an altmetric study on the most discussed articles on the web. Eur J Orthod 2018;40(2):193-199.
25. Ruan QZ, Chen AD, Cohen JB, Singhal D, Lin SJ, Lee BT. Alternative metrics of scholarly output: the relationship among altmetric score, reader score, citations, and downloads in plastic and reconstructive surgery. Plast Reconstr Surg 2018;141 (3):801 -809.
26. Schmid SL. Five years post-DORA: promoting best practices for research assessment. Mol Biol Cell 2017;28(22):2941 -2944.
27. Cagan R. The San Francisco declaration on research assessment. Dis Model Mech 2013;6(4):869 -870.
28. Peko S, Meyer H, Myssayeva K. You can't censor live: technology acceptance drives Central Asian journalists to mobile and helps them overcome press restrictions. Media Asia 2019;46(3-4):63-77.
ГЫЛЫМИ БАСЦАРУ YШIН К0РСЕТК1ШТЕР РЕТ1НДЕ ДЭЙЕКС0ЗДЕРГЕ НЕГ1ЗДЕЛГЕН
АЛМЕТРИКА ЖЭНЕ МЕТРИКА
ТYйiндеме
F ылыми макалалардын сапасы мен 9cepi дэстYрлi индикаторлардын KeMeriMeH багаланып, олардын арасында дэйексез саны Heri3ri болып табылады. Сандык дэуiрде желiлiк платформалар мен элеуметпк желтер гылыми акпаратты таратудын жетекшi арналарынын бiрiне айналды, сондыктан зертгеудi жоспарлау мен алынган гылыми нэтижелердi багалау Yшiн жана елшемдер пайда болды. Альтернативтi метрика (альтметрика) гылыми жумыстарды жедел багалау куралдары болып саналады олар: Интернетте жарияланганнан кейiн калай пайдаланылатындыгы жэне элеуметтiк желтер каналдарымен коса олардын кептеген элеуметтж платформалар аркылы таралатындыгы. Altmetric.com сайтынын Altmetrtic Attention Score багалаудын жаhандык маныздылыгы бар, ce6e6i бул жеке макалалардын Yнемi езгеретш элеуметтж эсерiн багалаудын интегралдык индикаторы болып табылады. Элеуметтiк желiлердiн осы жэне баска да кeрсеткiштерi белпленген дэйексез кeрсеткiштерiн едэуiр толыктырып, бул алгашкы сатысында жарияланганнан кейiн жарнамалык нэтижелердi керсетуге жэне болжамдауга мумкшдж бередi. Бул макалада эртYрлi альтметриялык куралдардын сипаттамалары жэне олардын гылыми зерттеулердi жоспарлау мен багалауда олардын релшщ артуы карастырылады.
ТYйiндi сездер: зерттеу, библиография такырып ретiнде/ элеуметтж медиа, альметрия, зейiндi альтметриялык багалау, Twitter, Mendeley
Дэйексез Yшiн: Кочийгит Б.Ф., Акёл А. Зерттеу менеджментшщ косымша кeрсеткiштерi ретiнде гi альтметрика жэне цитатика. Медициналык гипотеза мен этиканын Орта Азиялык журналы. 2021;2(2):79-84. https://doi.org/10.47316/caimhe.2021.2.2.01
АЛЬТМЕТРИКИ И МЕТРИКИ НА ОСНОВЕ ЦИТИРОВАНИЙ КАК КОМПЛЕМЕНТАРНЫЕ ИНДИКАТОРЫ ДЛЯ НАУЧНОГО МЕНЕДЖМЕНТА
Резюме
Качество и оказываемый эффект научных статей оценивается с помощью традиционных показателей, среди которых во главе угла стоит количество цитирований. В эпоху цифровых технологий онлайн-платформы и социальные сети стали одними из ведущих каналов распространения результатов научной информации, поэтому появились новые метрики для оценки планирования исследований и полученных научных результатов. Альтернативные метрики (альтметрики) являются инструментами для оперативной оценки научных работ: как они используются после публикации в Интернете и как распространяются через многочисленные онлайн-платформы, включая каналы социальных сетей. Оценка Altmetrtic Attention Score от Altmetric.com имеет глобальное значение, так как является интегральным индикатором для оценки постоянно меняющегося социального эффекта отдельных статей. Эти и другие показатели социальных сетей существенно дополняют установленные показатели цитируемости, позволяя визуализировать и прогнозировать результаты продвижения после публикации уже на ранних этапах. В этой статье рассматриваются характеристики различных альтметрических инструментов, их растущая роль в планировании и оценке научных исследований.
Ключевые слова: исследования, библиографии как тема, социальные сети, альтметрики, альтметрическая оценка внимания, Twitter, Mendeley
Для цитирования: Кочийгит Б.Ф., Акёл А. Альтметрика и цитатика как дополнительны интикаторы менеджмента исследований. Центральноазиатский журнал медицинских гипотез и этики. 2021;2(2):79-84. https://doi.org/10.47316/cajmhe.2021.2.2.01
