Vol. 14 No. 1 (2023): Peer review: a process undergoing a required transformation
Articles

Peer review in research assessment and data analysis of Italian publications in SSD M-STO/08 (Archival science, bibliography, library science)

Italian PDF
  • Rossana Morriello
Rossana Morriello
Politecnico di Torino
Bio
DOI: https://doi.org/10.36253/jlis.it-510

Published 2022-12-19

Keywords

  • Peer review,
  • Bibliometrics,
  • Research assessment,
  • VQR,
  • Academic field M-STO/08 (Archival science, bibliography and library science),
  • HSS,
  • Humanities and social sciences,
  • Academic research
    • ...More
    Less

How to Cite

Morriello, Rossana. 2022. “Peer Review in Research Assessment and Data Analysis of Italian Publications in SSD M-STO/08 (Archival Science, Bibliography, Library Science)”. JLIS.It 14 (1):99-120. https://doi.org/10.36253/jlis.it-510.

Copyright (c) 2022 Rossana Morriello

Creative Commons License

This work is licensed under a Creative Commons Attribution 4.0 International License.

Crossref
Scopus
Google Scholar
Europe PMC

Abstract

Since the introduction of research assessment systems at institutional level in the 1980s, the ongoing debate on the roles and functions of peer review and bibliometrics has been vivid and lively. In the first part of the article, the main lines over time of this debate are traced, and a reflection on the epistemic functions of peer review and citations is proposed. In Italy, the first research assessment exercise (VTR) was based on peer review only, while the following ones (VQR) were based on different methods for bibliometric disciplines and non-bibliometric disciplines, namely bibliometric indicators and peer review. Starting from a data analysis on Italian publications, and using as a sample data from M-STO/08 (Archival science, bibliography and library science) area, the essay shows some trends and changes in publication habits in HSS. Conclusions open a perspective on revitalization of peer review as a solid qualitative method for research assessment.

Italian PDF

Metrics

Metrics Loading ...

References

  1. Abramo, Giovanni. 2017. “Correspondence On tit for tat: Franceschini and Maisano versus ANVUR regarding the Italian research assessment exercise VQR 2011–2014”. Journal of Infometrics 11: 783-787. DOI: https://doi.org/10.1016/j.joi.2017.06.003
  2. Abramo, Giovanni, Ciriaco Andrea D’Angelo, e Alessandro Caprasecca. 2009. “Allocative efficiency in public research funding: Can bibliometrics help?”. Research Policy 38: 206-215. DOI: https://doi.org/10.1016/j.respol.2008.11.001
  3. ANVUR (Agenzia Nazionale di Valutazione del Sistema Universitario e della Ricerca). 2013. Rapporto finale sulla VQR 2004-2010. https://www.anvur.it/rapporto/.
  4. ANVUR (Agenzia Nazionale di Valutazione del Sistema Universitario e della Ricerca). 2018. Rapporto biennale sullo stato del sistema universitario e della ricerca. https://www.anvur.it/documenti-ufficiali/rapporti-sullo-stato/.
  5. Baccini, Alberto, e Giuseppe De Nicolao. 2016. “Do they agree? Bibliometric evaluation versus informed peer review in the Italian research assessment exercise”. Scientometrics 108: 1651–1671. DOI: https://doi.org/10.1007/s11192-016-1929-y
  6. Baldwin, Melinda. 2020. “Peer Review,” in Encyclopedia of the History of Science. Carnegie Mellon University. https://lps.library.cmu.edu/ETHOS/article/id/38/.
  7. Bence, Valerie e Charles Oppenheim. 2005. “The Evolution of the UK’s Research Assessment Exercise: Publications, Performance and Perceptions”. Journal of Educational Administration and History. 37 (2): 137-155. https://doi.org/10.1080/00220620500211189. DOI: https://doi.org/10.1080/00220620500211189
  8. Benedetto, Sergio, Daniele Checchi, Andrea Graziosi, e Marco Malgarini. 2017. “Comments on the paper Critical remarks on the Italian assessment exercise”. Journal of Informetrics, 11 (2): 622–624. DOI: https://doi.org/10.1016/j.joi.2017.03.005
  9. Bertocchi, Graziella, Alfonso Gambardella, Tullio Jappelli, Carmela A. Nappi, e Franco Peracchi. 2015. “Bibliometric evaluation vs. informed peer review: Evidence from Italy”. Research Policy 44 (2): 451-466. DOI: https://doi.org/10.1016/j.respol.2014.08.004
  10. Bulaitis, Zoe Hope. 2021. ““Minimum expectations” are no way to value the arts, humanities, and social sciences”. LSE Blog, June 7th. https://blogs.lse.ac.uk/impactofsocialsciences/2021/06/07/minimum-expectations-are-no-way-to-value-the-arts-humanities-and-social-sciences/.
  11. Bush, Vannevar. 2013. Manifesto per la rinascita di una nazione. Scienza, la frontiera infinita. Torino: Bollati Boringhieri.
  12. Castellucci, Paola. 2020. “La visione del giudizio. Una prospettiva romantica,” in Libri, biblioteche e società. Studi per Rosa Marisa Borraccini, a cura di Alberto Petrucciani, Valentina Sestini, Federico Valacchi. Macerata: EUM.
  13. Cronin, Blaise. 2005. The Hand of Science. Academic Writing and Its Rewards. Lanham: The Scarecrow Press.
  14. Demetrescu, Camil, Andrea Ribichini, e Marco Schaerf. 2020. “Are Italian research assessment exercises size‑biased?”. Scientometrics 125: 533-549. DOI: https://doi.org/10.1007/s11192-020-03643-x
  15. EUA (European University Association), Science Europe, Karen Stroobants, e (EC) European Commission. 2022. Agreement on Reforming Research Assessment. https://research-and-innovation.ec.europa.eu/news/all-research-and-innovation-news/reforming-research-assessment-agreement-now-final-2022-07-20_en.
  16. Faggiolani, Chiara. 2015. La bibliometria. Roma: Carocci.
  17. Franceschet, Massimo, e Antonio Costantini. “The first Italian research assessment exercise: A bibliometric perspective”. Journal of Infometrics 5: 275-291. DOI: https://doi.org/10.1016/j.joi.2010.12.002
  18. Franceschini, Fiorenzo, e Domenico Maisano. 2017. “Critical remarks on the Italian research assessment exercise VQR 2011–2014”. Journal of Infometrics 11: 337-357. DOI: https://doi.org/10.1016/j.joi.2017.02.005
  19. Galderisi, Claudio, Mauro Perretti, Nuria Sebastian Galles, e Thed van Leeuwen. 2019. Report of the Group of Experts charged by ANVUR to advice on the process ‘Valutazione della Qualità della Ricerca (VQR)’. An independent assessment on the past VQRs carried out by ANVUR. http://www.anvur.it/wp-content/uploads/2019/03/High-Experts-Report-on-VQR.pdf.
  20. Garfield, Eugene. 1973. “Citation Index depend upon the paper not the journal! Don’t count on ‘citation by association’”, Current Contents 22: 5-6. http://www.garfield.library.upenn.edu/essays/V1p452y1962-73.pdf.
  21. Garfield, Eugene. 1996. “How can impact factors be improved?” British Medical Journal, 313: 411–413, https://doi.org/10.1136/bmj.313.7054.411 DOI: https://doi.org/10.1136/bmj.313.7054.411
  22. Geuna, Aldo, e Matteo Piolatto. 2016. “Research assessment in the UK and Italy: Costly and difficult, but probably worth it (at least for a while)”. Research Policy 45 (1): 260-271. DOI: https://doi.org/10.1016/j.respol.2015.09.004
  23. Gingras, Yves. 2016. Bibliometrics and Research Evaluation: Uses and Abuses. Cambridge (Ma): MIT Press. DOI: https://doi.org/10.7551/mitpress/10719.001.0001
  24. Greco, Pietro. 2013. “Introduzione,” in V. Bush. Manifesto per la rinascita di una nazione. Scienza, la frontiera infinita. Torino: Bollati Boringhieri.
  25. Greco, Pietro. 2019. La scienza e l’Europa. Dal secondo dopoguerra a oggi. Roma: L’Asino D’Oro.
  26. Greenfield, Nathan M. 2022. “The unkindest cut? – Behind the paring of the humanities”. University World News. 13 August 2022. https://www.universityworldnews.com/post.php?story=20220808152834589.
  27. Guerrini, Mauro. 2022. “Sua Maestà il revisore: alcune considerazioni sul processo di peer-review all'interno della LIS”. AIB Studi, 61(3): 585–592. https://doi.org/10.2426/aibstudi-13328.
  28. Kaplan, Norman. 1965. “The Norms of Citation Behavior: Prolegomena to the Footnote”. American Documentation 16 (3): 179-184. DOI: https://doi.org/10.1002/asi.5090160305
  29. Meltzer, Leo. 1956. “Scientific Productivity in Organizational Settings”. Journal of Social Issues 12: 32-40. DOI: https://doi.org/10.1111/j.1540-4560.1956.tb00366.x
  30. Morriello, Rossana. 2019. “La valutazione della ricerca in Olanda. Un’analisi comparativa con il sistema italiano”. Biblioteche oggi 37 ottobre: 33-41.
  31. Myklebust, Jan Petter. 2021. “Copenhagen University set to cut humanities places by 24%”. University World News. 20 October. https://www.universityworldnews.com/post.php?story=20211020154347298.
  32. Phillimore, A.J. 1989. “University research performance indicators in practice: The University Grants Committee’s evaluation of British universities, 1985-86”. Research Policy 18: 255-171. DOI: https://doi.org/10.1016/0048-7333(89)90053-X
  33. Polanyi, Michael. 1962. “The Republic of Science: Its Political and Economic Theory”. Minerva I (1): 1-32. https://www.jstor.org/stable/41821153. DOI: https://doi.org/10.1007/BF01101453
  34. Polanyi, Michael. 2006. La società libera. Pensieri liberali, a cura di Massimo Baldini e Antonello Malavasi. Roma: Armando Editore.
  35. Popper, Karl. 2018. La società aperta e i suoi nemici. Roma: Armando Editore.
  36. Price, Derek de Solla. 1967. Sociologia della creatività scientifica. Milano, Bompiani.
  37. Pritchard, Alan. 1969. “Statistical bibliography or bibliometrics”. Journal of Documentation (25): 348-349. DOI: https://doi.org/10.1108/eb026482
  38. Rossi, Paolo. 2016. “Stato giuridico, reclutamento ed evoluzione della docenza universitaria (1975-2015)”. RT. A Journal on Research Policy & Evaluation 1. https://doi.org/10.13130/2282-5398/6726.
  39. Solimine, Giovanni. 2016. “Impatto e valutazione della ricerca scientifica”. Nuovi Annali della Scuola speciale per archivisti e bibliotecari. 30: 235-251.
  40. Solimine, Giovanni. 2017. Progetto LI.B.RO. For a Liable Evaluation of Book’s Role in Socio-Economic Sciences and Humanities: an International Comparison. Report finale. ANVUR, Working Paper 2017/06.
  41. Villa, Maria Luisa. 2018. Scienza è democrazia. Come funziona il mondo della ricerca. Postfazione di Pietro Greco. Milano: Guerini e Associati.
  42. VSNU (Association of Cooperating Universities in the Netherlands), KNAW (Royal Netherlands Academy of Arts and Sciences), e NWO (Dutch Research Council). 2020. Strategy Evaluation Protocol (SEP). https://www.universiteitenvannederland.nl/files/documenten/Domeinen/Onderzoek/SEP_2021-2027.pdf.
  43. Weingart, Peter. 2005. “Impact of bibliometrics upon the science system: Inadvertent consequences?”. Scientometrics 62 (1): 117-131. DOI: https://doi.org/10.1007/s11192-005-0007-7
  44. Wouters, Paul. 1998. “The Signs of Science”. Scientometrics 41 (1-2): 224-241. DOI: https://doi.org/10.1007/BF02457980
  45. Wouters, Paul. 2016. “Semiotics and Citations,” in Theories of Infometrics and Scholarly Communication. A Festschrift in honor of Blaise Cronin, edited by Cassidy R. Sugimoto. Berlin: De Gruyter.
  46. Zuckerman, Harriet, e Robert K. Merton. 1971. “Patterns of Evaluation in Science: Institutionalisation, Structure and Functions of the Referee System”. Minerva 9 (1): 66-100. DOI: https://doi.org/10.1007/BF01553188