Vol. 14 No. 1 (2023): Peer review: a process undergoing a required transformation
Articles

The role of peer review in the evaluation of research in Italy. Some remarks on the evaluation of PRINs

Italian PDF
  • Maurizio Vivarelli
Maurizio Vivarelli
University of Torino
Bio
DOI: https://doi.org/10.36253/jlis.it-500

Published 2022-12-19

Keywords

  • Peer review,
  • Research evaluation,
  • PRIN- Progetti di Rilevante Interesse Nazionale,
  • Academic field M-STO/08 (Archival science, bibliography and library science)

How to Cite

Vivarelli, Maurizio. 2022. “The Role of Peer Review in the Evaluation of Research in Italy. Some Remarks on the Evaluation of PRINs”. JLIS.It 14 (1):121-37. https://doi.org/10.36253/jlis.it-500.

Copyright (c) 2022 Maurizio Vivarelli

Creative Commons License

This work is licensed under a Creative Commons Attribution 4.0 International License.

Crossref
Scopus
Google Scholar
Europe PMC

Abstract

This contribution proposes some remarks on the evaluation and financing mechanisms of PRINs – Progetti di Rilevante Interesse Nazionale, promoted in Italy by the MUR - Ministry of University and Research, in the context of the critical issues and evolution prospects of peer review, of which a summary state of the art is presented. Starting from the partial and incomplete data made available on the MUR website dedicated to PRINs, are listed and examined the projects financed for the current disciplinary sector M-STO/08 (Archival Science, Bibliography and Librarianship), in the years between 1996 and 2020, and those included in other disciplinary areas that have as their subject matters related to the contents of the academic field M-STO/08.

Italian PDF

Metrics

Metrics Loading ...

References

  1. Batagelij, Vladimir, Anuška Ferligoj, e Flaminio Squazzoni. 2017. «The emergence of a field: a network analysis of research on peer review». Scientometrics 113: 503–532. https://doi.org/ 10.1007/s11192-017-2522-8. DOI: https://doi.org/10.1007/s11192-017-2522-8
  2. Biagetti, Maria Teresa. 2017. Valutare la ricerca nelle scienze umane e sociali: Potenzialità e limiti della Library catalog analysis, con testi di Antonella Iacono e Antonella Trombone. Milano: Editrice Bibliografica.
  3. Bonaccorsi, Andrea. 2020. «Two Decades of Research Assessment in Italy: Addressing the Criticisms». Scholarly Assessment Reports, 2 (1), 17. doi: http://doi.org/10.29024/sar.28. DOI: https://doi.org/10.29024/sar.28
  4. Brezis, Elise S., e Aliaksandr Birukou. 2020. «Arbitrariness in the Peer Review Process». Scientometrics 2123: 393–411. https://doi.org/10.1007/s11192-020-03348-1. DOI: https://doi.org/10.1007/s11192-020-03348-1
  5. Bunge, Mario. 1963. “A General Black Box Theory”. Philosophy of Science, 30 (4): 346-358. doi:10.1086/287954. DOI: https://doi.org/10.1086/287954
  6. Burke, Peter. 2002. Storia sociale della conoscenza: da Gutenberg a Diderot. Bologna: Il Mulino.
  7. Burke, Peter. 2013. Dall'Encyclopédie a Wikipedia: storia sociale della conoscenza. Bologna: Il Mulino.
  8. Castelvecchi, Davide. 2016. «Can we open the black box of AI? » Nature 538(7623): 20-23. doi: 10.1038/538020a. DOI: https://doi.org/10.1038/538020a
  9. Checco, Alessandro, Lorenzo Bracciale, Pierpaolo Loreti, Stephen Pinfield, e Giuseppe Bianchi. 2021. «AI-assisted peer review». Humanities and Social Sciences Communications 8(1): 1-11. doi: 10.1057/s41599-020-00703-8. DOI: https://doi.org/10.1057/s41599-020-00703-8
  10. Dondio, Pierpaolo, Niccolò Casnici, Francisco Grimaldo, Nigel Gilbert, e Flaminio Squazzoni. 2019. «The “Invisible Hand” of Peer Review: The Implications of Author-referee Networks on Peer Review in a Scholarly Journal». Journal of Informetrics 13: 708–716. doi: 10.1016/J.JOI.2019.03.018. DOI: https://doi.org/10.1016/j.joi.2019.03.018
  11. European Science Foundation. 2011. European Peer Review Guide. Integrating Policies and Practices into Coherent Procedures. https://www.esf.org/fileadmin/user_upload/esf/ European_Peer_Review _Guide_2011.pdf.
  12. Facchini Alessandro, e Alberto Termine. 2022. «Explainable AI: come andare oltre la black box degli algoritmi». Agenda digitale. https://www.agendadigitale.eu/cultura-digitale/explainable-ai-come-andare-oltre-la-black-box-degli-algoritmi/.
  13. Flaherty, Michael G. 2016. «Sociology as a Conversation: The Present Circumstances and Future Prospects of Peer Review». The American Sociologist 47:253–263. doi 10.1007/s12108-015-9299-0. DOI: https://doi.org/10.1007/s12108-015-9299-0
  14. García, José A., Rosa RodriguezSanchez, e Joaquin FdezValdivia. 2020. «Confirmatory Bias in Peer Review». Scientometrics 123: 517–533. https://doi.org/10.1007/s11192-020-03357-0. DOI: https://doi.org/10.1007/s11192-020-03357-0
  15. Grimaldo, Francisco, Ana Marušić, e Flaminio Squazzoni. 2018. «Fragments of Peer Review: A Quantitative Analysis of the Literature (1969-2015)». PLoS ONE 13(2): e0193148. https://doi.org/10.1371/journal.pone.0193148. DOI: https://doi.org/10.1371/journal.pone.0193148
  16. Heaven, Douglas. 2018. «The Age of AI Peer Reviews». Nature 563: 609-610. doi:10.1038/d41586-018-07245-9. DOI: https://doi.org/10.1038/d41586-018-07245-9
  17. Hjørland, Birger. 2018. «Library and Information Science (LIS)». ISKO Encyclopedia of Knowledge Organization, eds. Birger Hjørland e Claudio Gnoli, http://www.isko.org/cyclo/lis. DOI: https://doi.org/10.5771/0943-7444-2018-3-232
  18. Italia. Ministero dell’Università e della Ricerca. Decreto Direttoriale n. 104 del 02-02-2022. Bando Prin 2022. Linee guida per i revisori. https://www.mur.gov.it/sites/default/files/2022-02/DD%20n.%20104%20Linee%20Guida%20per%20i%20revisori%202022.pdf.
  19. Italia. Ministero dell’Università e della Ricerca. D.M. 30/10/2015 Rideterminazione dei macrosettori e dei settori concorsuali. http://attiministeriali.miur.it/anno-2015/ottobre/dm-30102015.aspx.
  20. Italia. Ministero dell’Università e della Ricerca. D.M. 4/10/2000 Settori scientifico-disciplinari. http://attiministeriali.miur.it/ anno-2000/ottobre/dm-04102000.aspx.
  21. Jefferson, Tom, Philip Alderson, Elizabeth Wager, e Frank Davidoff. 2002. «Effects of Editorial Peer Review: A Systematic Review». Journal of the American Medical Association 287 (21): 2784–2786. doi:10.1001/jama.287.21.2784. DOI: https://doi.org/10.1001/jama.287.21.2784
  22. Kuhn, Thomas 1962. The structure of scientific revolutions. Chicago: University of Chicago Press.
  23. Lee, Carole J., Cassidy R. Sugimoto, Guo Zhang, e Blaise Cronin. 2013. «Bias in Peer Review». Journal of the American Society for Information Science and Technology. 64: 2-17. https://doi.org/ 10.1002/asi.22784. DOI: https://doi.org/10.1002/asi.22784
  24. Morin, Edgar. 2001. I sette saperi necessari all’educazione del futuro. Milano: Raffaello Cortina: 11 (Le sept savoirs nécessaires à l’éducation du futur, 1999).
  25. Mrowinski, Maciej J., Piotr Fronczak, Agata Fronczak, Marcel Ausloos, e Olgica Nedic. 2017. «Artificial intelligence in peer review: How can evolutionary computation support journal editors?». PLoS ONE 12(9): e0184711. https://doi.org/10.1371/journal.pone.0184711. DOI: https://doi.org/10.1371/journal.pone.0184711
  26. Openpolis e AGI Italia. 2020. Ricerca e sviluppo 2020. L’impegno in innovazione di Italia e Ue rispetto all’obiettivo di Europa 2020. https://www.openpolis.it/wp-content/uploads/2020/04/Report-ricerca-e-sviluppo.pdf.
  27. Pascuzzi, Giovanni. 2012. «Una storia italiana: i settori scientifico-disciplinari», Materiali per una storia della cultura giuridica, 1: 91-122. doi: 10.1436/36843.
  28. Peer Review and the Acceptance of New Scientific Ideas. Discussion Paper from a Working Party on Equipping the Public with an Understanding of Peer Review, compiled and presented by Tracey Brown. 2004. London: Sense About Science. https://archive.senseaboutscience.org/data/files/resources/17/ peerReview.pdf.
  29. Phillips, Jonhaton P., Carina A. Hahn, Peter C. Fontana, Amy N. Yates, Kristen Greene, David A. Broniatowski, e Mark A. Przybocki. 2021. Four Principles of Explainable Artificial Intelligence. NIST Interagency/Internal Report (NISTIR), National Institute of Standards and Technology, Gaithersburg, MD. https://doi.org/10.6028/NIST.IR.8312, https://tsapps.nist.gov/publication /get_pdf.cfm?pub_id=933399. DOI: https://doi.org/10.6028/NIST.IR.8312
  30. Rigby, John, Jakob Edler. 2005. «Peering Inside Research Networks: Some Observations on the Effect of the Intensity of Collaboration on the Variability of Research Quality». Research Policy 34, 6: 784-794. https://doi.org/10.1016/j.respol.2005.02.004. DOI: https://doi.org/10.1016/j.respol.2005.02.004
  31. Ross-Hellauer, Tony. 2017. «What is Open Peer Review? A Systematic Review». F1000Res. 6:588. doi: 10.12688/f1000research.11369.2. PMID: 28580134; PMCID: PMC5437951. DOI: https://doi.org/10.12688/f1000research.11369.2
  32. Serrai, Alfredo. 1973. Biblioteconomia come scienza: introduzione ai problemi ed alla metodologia. Firenze: Olschki.
  33. Squazzoni, Flaminio, Elise Brezis, e Ana Marušić. 2017. «Scientometrics of Peer Review». Scientometrics 113 (1): 501-502. doi: 10.1007/s11192-017-2518-4. DOI: https://doi.org/10.1007/s11192-017-2518-4
  34. Vincent-Lamarre, Philippe, e Vincent Larivière. 2021. «Textual Analysis of Artificial Intelligence Manuscripts Reveals Features Associated with Peer Review Outcome». Quantitative Science Studies, 2(2): 662-677. https://doi.org/10.1162/qss_a_00125. DOI: https://doi.org/10.1162/qss_a_00125