Analysis of Effects on Scientific Impact Indicators Based on Coevolution of Coauthorship and Citation Networks
Abstract
:1. Introduction
1.1. Research Background and Significance
1.2. Literature Review
1.2.1. Citation Network
1.2.2. Coauthorship Network
1.2.3. Coevolution of Both Networks
1.2.4. Scientific Impact Indicators
1.3. Theoretical and Practical Implications
- 1.
- Theoretical Implications:
- 2.
- Practical Implications:
2. Model Formulation and Validation
2.1. APS Database
2.2. Growth of Number of Papers and Number of Authors
- The probability that the − st word is a word that has already appeared exactly times is proportional to —that is, to the total numbers of occurrences of all the words that have appeared exactly times.
- There is a constant probability, , that the − st word be a new word—a word that has not occurred in the first words.
2.3. Paper Team Assembly
2.4. Author Ability and Paper Quality
2.5. Coauthorship Network
2.6. Reference Model
2.7. Citation Network
2.8. Journal Impact Factor
2.9. h-Index
2.10. Rationale Behind Parameter Choices and Model Architecture
- 1.
- Model Architecture and Parameterization:
- 2.
- Impact of Parameters:
2.11. Comparison with State-of-the-Art (SOTA) Baseline Models
2.12. Generalizability across Disciplines
3. Sensitivity Analysis and Results
3.1. Paper Lifetime θ
3.2. Reference Number
3.3. Team Size m at Fixed p
3.4. Probability of Newcomers p
3.5. Team Size m at Fixed k
4. Discussion
4.1. Interpretation of Findings in Relation to Previous Literature
4.2. Actionable Insights for Researchers, Publishers, and Policymakers
- 1.
- For Researchers:
- 2.
- For Publishers:
- 3.
- For Policymakers:
4.3. Ethical Considerations in the Use of Scientific Impact Indicators
- 1.
- Manipulation of Metrics:
- 2.
- The Role of Institutions and Publishers:
- 3.
- Promoting Ethical Research Practices:
4.4. Limitations of the Study
- 1.
- Disciplinary Specificity:
- 2.
- Basic Citation Impact Indicators:
- 3.
- Ethical Concerns and Metric Manipulation:
4.5. Future Research Directions
- 1.
- Inclusion of Field- and Time-Normalized Indicators:
- 2.
- Cross-Disciplinary Validation:
- 3.
- Development of Ethical Evaluation Metrics:
- 4.
- Integration with Emerging Metrics:
5. Conclusions
- By using a few simple and reasonable assumptions, the mathematical models can effectively replicate most empirical data characteristics, including temporal dynamics and distributions of -index, thus indicating that modeling and simulation methods are reliable tools for exploring how different parameters affect scientific impact indicators.
- Increasing the reference number or decreasing the paper lifetime significantly boosted both the journal impact factor and average -index. Additionally, enlarging team size without adding new authors or reducing the probability of selecting newcomers notably increases the average -index. This implies that scientific impact indicators may have inherent weaknesses or can be manipulated by authors, making them unreliable for assessing the true quality of a paper.
- The presented mathematical models can be easily extended to include other scientific impact indicators and scenarios. This versatility positions modeling and simulation methods as powerful tools for studying the impact of various parameters on scientific impact indicators, aiding in the development of improved indicators. Furthermore, these methods can serve as robust tools for validating underlying mechanisms or predicting different scenarios based on joint coauthorship and citation networks.
Supplementary Materials
Funding
Institutional Review Board Statement
Informed Consent Statement
Data Availability Statement
Conflicts of Interest
References
- Su, F.; Li, S.; Liu, Q.; Zhang, Y. A bibliometric study of digital humanities research in China from 2012 to 2021. Inf. Res. Int. Electron. J. 2023, 28, 62–82. [Google Scholar] [CrossRef]
- Geraldo, G.; Bisset-Alvarez, E.; Pinto, M.D.d.S. Digital Humanities and the Sustainable Development Goals: A reflection for Information Science. Transinformação 2023, 35, e227210. [Google Scholar] [CrossRef]
- Osinska, V.; Klimas, R. Mapping science: Tools for bibliometric and altmetric studies. Inf. Res. 2021, 26, 909. [Google Scholar] [CrossRef]
- Yan, C.; Li, H.; Pu, R.; Deeprasert, J.; Jotikasthira, N. Knowledge mapping of research data in China: A bibliometric study using visual analysis. Libr. Hi Tech. 2024, 42, 331–349. [Google Scholar] [CrossRef]
- Dong, J.Q. Using simulation in information systems research. J. Assoc. Inf. Syst. 2022, 23, 408–417. [Google Scholar] [CrossRef]
- Medo, M.; Cimini, G. Model-based evaluation of scientific impact indicators. Phys. Rev. E 2016, 94, 032312. [Google Scholar] [CrossRef] [PubMed]
- Bai, X.; Zhang, F.; Li, J.; Xu, Z.; Patoli, Z.; Lee, I. Quantifying scientific collaboration impact by exploiting collaboration-citation network. Scientometrics 2021, 126, 7993–8008. [Google Scholar] [CrossRef]
- Scharnhorst, A.; Börner, K.; Van den Besselaar, P. Models of Science Dynamics: Encounters between Complexity Theory and Information Sciences; Springer Science & Business Media: Berlin/Heidelberg, Germany, 2012. [Google Scholar]
- Zhang, H.; Guo, J.; Guo, F.; Zhang, W. Knowledge Networks, Collaboration Networks, and Local Search Behaviors. Group Organ. Manag. 2023, 10596011231203364. [Google Scholar] [CrossRef]
- Price, D.d.S. A general theory of bibliometric and other cumulative advantage processes. J. Am. Soc. Inf. Sci. 1976, 27, 292–306. [Google Scholar] [CrossRef]
- Barabási, A.-L.; Albert, R. Emergence of scaling in random networks. Science 1999, 286, 509–512. [Google Scholar] [CrossRef] [PubMed]
- Bianconi, G.; Barabási, A.-L. Competition and multiscaling in evolving networks. Europhys. Lett. 2001, 54, 436. [Google Scholar] [CrossRef]
- Medo, M.; Cimini, G.; Gualdi, S. Temporal effects in the growth of networks. Phys. Rev. Lett. 2011, 107, 238701. [Google Scholar] [CrossRef] [PubMed]
- Eom, Y.-H.; Fortunato, S. Characterizing and modeling citation dynamics. PLoS ONE 2011, 6, e24926. [Google Scholar] [CrossRef] [PubMed]
- Wang, D.; Song, C.; Barabási, A.-L. Quantifying long-term scientific impact. Science 2013, 342, 127–132. [Google Scholar] [CrossRef] [PubMed]
- Wang, D.; Barabási, A.-L. The Science of Science; Cambridge University Press: Cambridge, UK, 2021. [Google Scholar]
- Song, D.; Wang, W.; Fan, Y.; Xing, Y.; Zeng, A. Quantifying the structural and temporal characteristics of negative links in signed citation networks. Inf. Process. Manag. 2022, 59, 102996. [Google Scholar] [CrossRef]
- Radicchi, F.; Fortunato, S.; Castellano, C. Universality of citation distributions: Toward an objective measure of scientific impact. Proc. Natl. Acad. Sci. USA 2008, 105, 17268–17272. [Google Scholar] [CrossRef]
- Brunton, F. Making people and influencing friends: Citation networks and the appearance of significance. In Gaming the Metrics: Misconduct and Manipulation in Academic Research; The MIT Press: Cambridge, MA, USA, 2020; p. 243. [Google Scholar]
- Herteliu, C.; Ausloos, M.; Ileanu, B.V.; Rotundo, G.; Andrei, T. Quantitative and qualitative analysis of editor behavior through potentially coercive citations. Publications 2017, 5, 15. [Google Scholar] [CrossRef]
- Yousefi Nooraie, R.; Sale, J.E.; Marin, A.; Ross, L.E. Social network analysis: An example of fusion between quantitative and qualitative methods. J. Mix. Methods Res. 2020, 14, 110–124. [Google Scholar] [CrossRef]
- Bologna, F.; Di Iorio, A.; Peroni, S.; Poggi, F. Do open citations give insights on the qualitative peer-review evaluation in research assessments? An analysis of the Italian National Scientific Qualification. Scientometrics 2023, 128, 19–53. [Google Scholar] [CrossRef]
- Newman, M.E. Coauthorship networks and patterns of scientific collaboration. Proc. Natl. Acad. Sci. USA 2004, 101 (Suppl. 1), 5200–5205. [Google Scholar] [CrossRef] [PubMed]
- Barabâsi, A.-L.; Jeong, H.; Néda, Z.; Ravasz, E.; Schubert, A.; Vicsek, T. Evolution of the social network of scientific collaborations. Phys. A Stat. Mech. Its Appl. 2002, 311, 590–614. [Google Scholar] [CrossRef]
- Newman, M.E. Scientific collaboration networks. I. Network construction and fundamental results. Phys. Rev. E 2001, 64, 016131. [Google Scholar] [CrossRef] [PubMed]
- Newman, M.E. The structure of scientific collaboration networks. Proc. Natl. Acad. Sci. USA 2001, 98, 404–409. [Google Scholar] [CrossRef] [PubMed]
- Tomassini, M.; Luthi, L. Empirical analysis of the evolution of a scientific collaboration network. Phys. A Stat. Mech. Its Appl. 2007, 385, 750–764. [Google Scholar] [CrossRef]
- Wuchty, S.; Jones, B.F.; Uzzi, B. The increasing dominance of teams in production of knowledge. Science 2007, 316, 1036–1039. [Google Scholar] [CrossRef] [PubMed]
- Guimera, R.; Uzzi, B.; Spiro, J.; Amaral, L.A.N. Team assembly mechanisms determine collaboration network structure and team performance. Science 2005, 308, 697–702. [Google Scholar] [CrossRef] [PubMed]
- Twyman, M.; Contractor, N. Team assembly. In Strategies for Team Science Success: Handbook of Evidence-Based Principles for Cross-Disciplinary Science and Practical Lessons Learned from Health Researchers; Springer: Cham, Switzerland, 2019; pp. 217–240. [Google Scholar]
- Yang, J.; Wu, L.; Lyu, L. Research on scientific knowledge evolution patterns based on ego-centered fine-granularity citation network. Inf. Process. Manag. 2024, 61, 103766. [Google Scholar] [CrossRef]
- Pessoa Junior, G.J.; Dias, T.M.; Silva, T.H.; Laender, A.H. On interdisciplinary collaborations in scientific coauthorship networks: The case of the Brazilian community. Scientometrics 2020, 124, 2341–2360. [Google Scholar] [CrossRef]
- Orzechowski, K.P.; Mrowinski, M.J.; Fronczak, A.; Fronczak, P. Asymmetry of social interactions and its role in link predictability: The case of coauthorship networks. J. Informetr. 2023, 17, 101405. [Google Scholar] [CrossRef]
- Pelacho, M.; Ruiz, G.; Sanz, F.; Tarancón, A.; Clemente-Gallardo, J. Analysis of the evolution and collaboration networks of citizen science scientific publications. Scientometrics 2021, 126, 225–257. [Google Scholar] [CrossRef]
- Chen, W.; Yan, Y. New components and combinations: The perspective of the internal collaboration networks of scientific teams. J. Informetr. 2023, 17, 101407. [Google Scholar] [CrossRef]
- Hâncean, M.-G.; Perc, M.; Lerner, J. The coauthorship networks of the most productive European researchers. Scientometrics 2021, 126, 201–224. [Google Scholar] [CrossRef]
- Singh, C.K.; Filho, D.V.; Jolad, S.; O’Neale, D.R. Evolution of interdependent co-authorship and citation networks. Scientometrics 2020, 125, 385–404. [Google Scholar] [CrossRef]
- Börner, K.; Maru, J.T.; Goldstone, R.L. The simultaneous evolution of author and paper networks. Proc. Natl. Acad. Sci. USA 2004, 101 (Suppl. S1), 5266–5273. [Google Scholar] [CrossRef]
- Xie, Z.; Xie, Z.; Li, M.; Li, J.; Yi, D. Modeling the coevolution between citations and coauthorship of scientific papers. Scientometrics 2017, 112, 483–507. [Google Scholar] [CrossRef]
- Zhang, X.; Xie, Q.; Song, M. Measuring the impact of novelty, bibliometric, and academic-network factors on citation count using a neural network. J. Informetr. 2021, 15, 101140. [Google Scholar] [CrossRef]
- Liu, X.F.; Chen, H.-J.; Sun, W.-J. Adaptive topological coevolution of interdependent networks: Scientific collaboration-citation networks as an example. Phys. A Stat. Mech. Its Appl. 2021, 564, 125518. [Google Scholar] [CrossRef]
- Muñoz-Muñoz, A.M.; Mirón-Valdivieso, M.D. Analysis of collaboration and co-citation networks between researchers studying violence involving women. Inf. Res. Int. Electron. J. 2017, 22, n2. [Google Scholar]
- Hirsch, J.E. An index to quantify an individual’s scientific research output. Proc. Natl. Acad. Sci. USA 2005, 102, 16569–16572. [Google Scholar] [CrossRef] [PubMed]
- Garfield, E. Citation analysis as a tool in journal evaluation: Journals can be ranked by frequency and impact of citations for science policy studies. Science 1972, 178, 471–479. [Google Scholar] [CrossRef] [PubMed]
- Zhou, J.; Cai, N.; Tan, Z.-Y.; Khan, M.J. Analysis of effects to journal impact factors based on citation networks generated via social computing. IEEE Access 2019, 7, 19775–19781. [Google Scholar] [CrossRef]
- Zhou, J.; Feng, L.; Cai, N.; Yang, J. Modeling and simulation analysis of journal impact factor dynamics based on submission and citation rules. Complexity 2020, 2020, 3154619. [Google Scholar] [CrossRef]
- Guns, R.; Rousseau, R. Simulating growth of the h-index. J. Am. Soc. Inf. Sci. Technol. 2009, 60, 410–417. [Google Scholar] [CrossRef]
- Ionescu, G.; Chopard, B. An agent-based model for the bibliometric h-index. Eur. Phys. J. B 2013, 86, 426. [Google Scholar] [CrossRef]
- Bornmann, L.; Marx, W.; Gasparyan, A.Y.; Kitas, G.D. Diversity, value and limitations of the journal impact factor and alternative metrics. Rheumatol. Int. 2012, 32, 1861–1867. [Google Scholar] [CrossRef]
- Harzing, A.-W.; Alakangas, S. Google Scholar, Scopus and the Web of Science: A longitudinal and cross-disciplinary comparison. Scientometrics 2016, 106, 787–804. [Google Scholar] [CrossRef]
- McKiernan, E.C.; Schimanski, L.A.; Muñoz Nieves, C.; Matthias, L.; Niles, M.T.; Alperin, J.P. Use of the Journal Impact Factor in academic review, promotion, and tenure evaluations. eLife 2019, 8, e47338. [Google Scholar] [CrossRef]
- Fire, M.; Guestrin, C. Over-optimization of academic publishing metrics: Observing Goodhart’s Law in action. GigaScience 2019, 8, giz053. [Google Scholar] [CrossRef] [PubMed]
- Chapman, C.A.; Bicca-Marques, J.C.; Calvignac-Spencer, S.; Fan, P.; Fashing, P.J.; Gogarten, J.; Guo, S.; Hemingway, C.A.; Leendertz, F.; Li, B. Games academics play and their consequences: How authorship, h-index and journal impact factors are shaping the future of academia. Proc. R. Soc. B 2019, 286, 20192047. [Google Scholar] [CrossRef] [PubMed]
- Waltman, L. A review of the literature on citation impact indicators. J. Informetr. 2016, 10, 365–391. [Google Scholar] [CrossRef]
- Sinatra, R.; Wang, D.; Deville, P.; Song, C.; Barabási, A.-L. Quantifying the evolution of individual scientific impact. Science 2016, 354, aaf5239. [Google Scholar] [CrossRef] [PubMed]
- American Physical Society. APS Data Sets for Research. 2023. Available online: https://journals.aps.org/datasets (accessed on 29 July 2024).
- Simon, H.A. On a class of skew distribution functions. Biometrika 1955, 42, 425–440. [Google Scholar] [CrossRef]
- Ijiri, Y.; Simon, H.A. Skew Distributions and the Sizes of Business Firms. R. Stat. Soc. J. Ser. A Gen. 1977, 140, 547–548. [Google Scholar] [CrossRef]
- Simon, H.A.; Van Wormer, T.A. Some Monte Carlo estimates of the Yule distribution. Behav. Sci. 1963, 8, 203–210. [Google Scholar] [CrossRef]
- Milojević, S. Principles of scientific research team formation and evolution. Proc. Natl. Acad. Sci. USA 2014, 111, 3984–3989. [Google Scholar] [CrossRef]
- Bornmann, L. Is collaboration among scientists related to the citation impact of papers because their quality increases with collaboration? An analysis based on data from F1000Prime and normalized citation scores. J. Assoc. Inf. Sci. Technol. 2017, 68, 1036–1047. [Google Scholar] [CrossRef]
- Ahlgren, P.; Colliander, C.; Sjögårde, P. Exploring the relation between referencing practices and citation impact: A large-scale study based on Web of Science data. J. Assoc. Inf. Sci. Technol. 2018, 69, 728–743. [Google Scholar] [CrossRef]
- Fortunato, S.; Bergstrom, C.T.; Börner, K.; Evans, J.A.; Helbing, D.; Milojević, S.; Petersen, A.M.; Radicchi, F.; Sinatra, R.; Uzzi, B. Science of science. Science 2018, 359, eaao0185. [Google Scholar] [CrossRef]
- Golosovsky, M. Universality of citation distributions: A new understanding. Quant. Sci. Stud. 2021, 2, 527–543. [Google Scholar] [CrossRef]
- Liu, P.; Xia, H. Structure and evolution of co-authorship network in an interdisciplinary research field. Scientometrics 2015, 103, 101–134. [Google Scholar] [CrossRef]
- Waltman, L.; van Eck, N.J.; van Raan, A.F. Universality of citation distributions revisited. J. Am. Soc. Inf. Sci. Technol. 2012, 63, 72–77. [Google Scholar] [CrossRef]
Disclaimer/Publisher’s Note: The statements, opinions and data contained in all publications are solely those of the individual author(s) and contributor(s) and not of MDPI and/or the editor(s). MDPI and/or the editor(s) disclaim responsibility for any injury to people or property resulting from any ideas, methods, instructions or products referred to in the content. |
© 2024 by the author. Licensee MDPI, Basel, Switzerland. This article is an open access article distributed under the terms and conditions of the Creative Commons Attribution (CC BY) license (https://creativecommons.org/licenses/by/4.0/).
Share and Cite
Xue, H. Analysis of Effects on Scientific Impact Indicators Based on Coevolution of Coauthorship and Citation Networks. Information 2024, 15, 597. https://doi.org/10.3390/info15100597
Xue H. Analysis of Effects on Scientific Impact Indicators Based on Coevolution of Coauthorship and Citation Networks. Information. 2024; 15(10):597. https://doi.org/10.3390/info15100597
Chicago/Turabian StyleXue, Haobai. 2024. "Analysis of Effects on Scientific Impact Indicators Based on Coevolution of Coauthorship and Citation Networks" Information 15, no. 10: 597. https://doi.org/10.3390/info15100597
APA StyleXue, H. (2024). Analysis of Effects on Scientific Impact Indicators Based on Coevolution of Coauthorship and Citation Networks. Information, 15(10), 597. https://doi.org/10.3390/info15100597