Towards peer review as a group engagement

I discuss from an economic perspective two of the most recent suggestions to reform the peer review system: (a) payment to referees; (b) ex post peer review. I show that strong economic arguments militate against these ideas. With respect to payment to referees I use results from the economic analysis of prosocial behavior and the private production of public goods, which show that the supply of monetary incentives has the paradoxical effect of reducing the willingness of agents to collaborate, insofar as they substitute intrincic motivation with extrinsic motivation. With respect to ex post peer review, I show that it fails to offer sufficient incentives to researchers, since it is anonymous, depersonalized, and weak in its marginal impact on publishing decisions. I take this argument to criticize the lack of the-orizing, in the side of radical proponents of Open access, about the conditions for transition from the subscription model to the Open model. It is this lack of critical attention to economic arguments that has led to the unintended but dramatic outcome of a net increase in the cost of scientific publishing, as documented in very recent papers. Finally, I advance a proposal for admitting payments to referees, but not as individuals but as groups of researchers. I offer this idea to open discussion.


Introduction
The existence and functioning of peer review is an issue that has attracted the attention of economists since long time. Peer review is a complex system which works almost entirely without a price mechanism. Referees donate their time without compensation. According to Aczel et al. (2021) the total time reviewers globally worked on peer reviews was over 100 million hours in 2020, equivalent to over 15 thousand years, or a monetary value of 1.5 billion dollar in USA and 600 million dollar in China. In a very recent correspondence in Human Genomics Reichardt et al. (2022) have protested against the introduction by Springer of a new software for the management of editorial processes without any consultation with referees. They argue that "highly skilled but unpaid volunteer workers should neither be taken for granted nor forgotten altogether" and warn against the possibility that editors and reviewers "go on strike" (Reichardt et al. 2022, 1-2), as a consequence of unsustainable "evaluator fatigue" (Golden and Schultz 2012;Breuning et al. 2015). In this paper I examine two suggestions that have been advanced to address the overload, namely: (a) paying individual referees; and (b) shifting from ex ante to ex post peer review. I suggest that in both cases the proponents failed to carry out a comprehensive economic analysis of the potential consequences. This omission, however, is asymmetric. In the case of payment to referees it must be said that its implementation is extremely rare, so that that impact of the proposal on the scientific system has been negligible. In the case of ex post review, on the contrary, I will show a perverse effect. This proposal has shifted the attention of the academic community and of policy makers away from a rigorous analysis of the consequences of the introduction on a large scale of the Open access model. In particular, it has made the community blind with respect to the unintended but extremely serious outcome that has been recently denounced by Zhang et al. (2022), namely, that the cost of publishing has actually increased after the diffusion of Open access. The overall research system has become more closed, not more open. Finally, I formulate a modest proposal for improving the situation-shifting the burden of peer review from isolated individuals to groups and provide collective, not individual, payment. The discussion is mainly conceptual and preliminary.

Should we pay the referees? On the economics of unpaid peer review work
The peer review system is clearly under pressure. The risk is that in the near future the system will not be able to deliver the expected outcome either in quantitative (i.e. processing the require volume of submissions) as well as qualitative terms (i.e. ensuring careful review of submissions). On the volume side of the problem, the system suffers under the weight of its own success. The number of scientific articles published each year continues to grow. This is not only due to the rise in the number of authors, but also to the increase in average productivity, facilitated by improvements in research technologies, either at laboratory level and in the management of scientific information. As a result, researchers produce more publications and at the same time are asked to review a larger number of publications per year. The system has become too slow (Hauser and Fehr 2007). On a different dimension, peer review is under pressure as a result of a movement of critical studies that have shown several types of distortions (Smith 2006;Schroter 2007;Tennant et al. 2020), including biased reviews and inconsistency (see Seeber 2022 for a careful survey). JLIS.it vol. 14,no.  The quantitative and qualitative dimensions of the peer review system, although different in the main causal factors, are clearly interdependent. Given a (relatively) fixed time budget, increasing the number of referee reports means allocating a smaller number of hours to each of them. More subtly, if researchers allocate a fixed proportion of their time to referee work, it is likely that they will accept submissions in a hierarchical order, first from journals of higher reputation. If they saturate their time, they will refuse submissions from lower-tier journals. If this dynamics is replicated, the final result will be that lower-tier journals will make appeal to researchers that allocate a larger share of their time to referee work. It is in this population that we might expect to find referees that "cut the corners" of their reports in order to save time. Under this scenario, the main issue is not that the supply of referee work does not match the demand, but that the overall quality deteriorates (Golden and Schultz 2012;Kovanis et al. 2016).
The questions is what kind of changes in the peer review system are required in order to preserve the sustainability, that is, to obtain a sufficiently large volume of good quality reports. The current system seems to survive only by miracle (Righi and Takacs 2017). Several solutions are under experimentation (Walker and Rocha da Silva 2015). Several authors have recently suggested that, in order to preserve the sustainability of the system, it would be a good idea to offer economic incentives to referees, in the form of payment (Capiello 2018; Frjsters and Torgler 2019; García et al. 2021;Cheah and Piasecki 2022), or in the form of penalties for lack of compliance (Hauser and Fehr 2007). This idea is not new. It has been repeatedly advanced in the literature, with slight variations (Fox and Petchey 2010;Ott and Hebenstreit 2014;Diamandis 2015). But is receiving new attention, after the prestigious medical journal Lancet opened a debate. In the correspondence following Cheah and Piasecki's (2022) piece several authors supported the idea (Humphreys 2022;Chalmers and Solomon 2022;Guinart et al. 2022) and a few opposed it (Moustafa 2022). As an argument, Cheah and Piasecki (2022) argue that in the biomedical field the pool of referees is formed mainly by researchers in rich countries. This is because researchers in poor countries do not have time to undertake unpaid work. The unpleasant result is that the overall agenda of biomedical research might be biased by the health problems of rich countries (say, obesity) rather than of other countries. Offering small economic incentives to referees might address this problem and rebalance the pool of referees. García et al. (2021) introduced a new argument in favour of the idea: there is information asymmetry between editors and referees, so that editors do not actually know whether referees put adequate effort in their job. Given this moral hazard situation, paying referees would be the best solution to avoid deterioration of quality. From an economic perspective, all kinds of unpaid work require an explanation. Consequently, all human actions that take place without an explicit reward have been carefully examined in the literature: from blood donation to donations to charities, from the private provision of public goods to voluntary activities, from the production of Open source software to peer review. The overall analytical framework is somewhat similar: given the absence of monetary incentives, economists try to identify (sometimes hidden) non-monetary forms of compensation. In particular, the theoretical issue is to explain why economic agents engage into behaviors that, under the assumptions of rationality commonly accepted in economics, should be considered irrational (Della Vigna et al. 2012). Why do software programmers write code for free under the Open source movement, when they might produce the same code and being paid? Why do people ISSN: 2038-1026 online Open access article licensed under CC-BY DOI: 10.36253/jlis.it-511 contribute to public goods (e.g. pay for a public city facility) even when they are not forced to do so, are not controled, and do not see any direct advantage? With these behaviors people are producing public goods, that is, create externalities that benefit other and not themselves. This creates a dilemma. All these issues are extensively examined in public economics and, more recently, form a large stream of studies in experimental economics and behavioral economics. A commonly accepted argument is that people receive some reward from non-self-interested behavior due to the existence of social norms, or conventions, or shared beliefs about the social acceptability and normativity of a prosocial behavior. In particular, a prominent explanation is offered by the psychological notion of intrinsic motivation. The starting point is the opposition between intrinsic and extrinsic motivations suggested in social psychology (Deci 1971;Deci et al. 1999;Ryan and Deci 2000). Intrinsic motivation does not depend on monetary incentives but is generated by personal beliefs of economic agents, or their self-determination. Extrinsic motivation, on the contrary, is significantly influenced by the offering of rewards, mostly in monetary terms. People who act on the basis of extrinsic motivations will change their behavior following the external provision of incentives. The issue whether economic incentives destroy intrinsic motivation has been examined at great lenght. A controversial issue is what happens when extrinsic motivations are added to intrinsic motivation, or even when they are substituted for. An influential research tradition has shown that extrinsic motivation has a crowding effect on the intrinsic one (Frey and Jegen 2001;Kamenica 2012). Pioneering experiments on the donation of blood have shown that offering cash incentives to donors actually decreases the overall amount of donations. There is an overwhelming evidence that adding monetary incentives, or substituting monetary incentives for non-monetary ones, produces a crowding out effect (James 2005; Festré and Garrouste 2014). 1 Squazzoni and Gandelli (2012), Squazzoni et al. (2013) and Zaharie and Seeber (2018) examined the issue of crowding out in the context of peer review and largely confirmed that monetary rewards do not increase the probability of acceptance and may even induce the perverse effect of deteriorating the quality of referee reports. This line of thinking has been followed in the field of peer review by in a carefully designed experiment (Chetty et al. 2014). The authors asked a pool of 1,500 referees of the Journal of Public Economics to review a paper. Referees were assigned to four experimental conditions: the first group was asked to deliver the report in six weeks, the second group in four weeks, while the third was offered a payment of 100 USD for meeting the four week deadline and a final group was informed that their turnaround times would be made public (social pressure). What they find is that cash incentives significantly improve speed and do not crowd out intrinsic motivation, since referees continue delivering reports in four weeks even after the end of the incentive treatment. They suggest that price incentives are complementary to social pressure. The results support the notion that offering monetary incentives does have a positive effect, but the generalizability of the study to disciplnes other than economics is not obvious. According to a survey in the biomedical JLIS.it vol. 14, no. 1 (January 2023) ISSN: 2038-1026 online Open access article licensed under CC-BY DOI: 10.36253/jlis.it-511 field, for example, (Tite and Schroter 2007) "most respondents agreed that financial incentives would not be effective when time constraints are prohibitive". Since the evidence for crowding out is overwhelming, I would be reluctant to introduce monetary payment in the referee work. Intrinsic motivation is a crucial component of the researcher identity. As shown by Stern (2004), researchers in public institutions deliberately accept a lower salary with respect to people of comparable seniority and postgraduate education employed in the private sector, in exchange for more autonomy and freedom in the organization of their work. Peer review is felt as an obligation to the scientific community which is integral part of the profession (Trevino 2008; Northcraft and Tenbrunsel 2011). Autonomy and freedom are not self-produced. They are the result of the organization of science as a community. Outside the interdependence among researchers there is no way to validate scientific output. Providing referee work for free is still considered, and rightly so, an essential part of belonging to a community. The problem with peer review is not the lack of economic incentives, it is the overload, on the one hand, and the perception of unfairness with respect to the private business models, on the other hand (see below for this issue). Offering economic incentives would provide only a marginal increase in the referee work. But it would destroy the ethos of scientific communities. Another problem lies not in the incentive effect per se, but in its size. A number of influential studies have shown that, if the incentive is not sufficiently large, people do not change their behavior (Gneezy and Rustichini 2000;Gneezy et al. 2011). This may be a reason why performance incentives in work systems sometimes fail, particularly in environments that require creative activities (Gerhart and Fang 2015). As far as I know this argument has not been taken into account by proponents of payment. Summing up, I do not consider the payment of individual referees appropriate.

The myth of ex post review and the unintended consequences of Open access
Another solution for the overload of peer review has been the shifting of the referee work from the ex ante review (before publication) to the ex post review (after publication in open archives). This idea has been advanced in the context of the movement that has suggested that Open access should be the only way of publication of scientific output (Hunter 2012;Kriegeskorte 2012). This suggestion comes as a "first publish then evaluate" model. My interpretation of the issue is as follows. The proponents of the Open access model have deliberately neglected an economic analysis of the incentive system of science. In various occasions the suggestion of Open access has been associated to the notion of ex post peer review-the idea that all publications should be openly available in public platforms, such as university open archives, and be subject to peer review by all interested parties, after the publication. This idea was associated to a critique of the traditional peer review system, according to which the selection of referees (it does not matter whether under a double blind, single blind, or transparent model) by editors introduces distortions in the directions of science. According to a Special issue of Frontiers in Computational Neuroscience  there is large consensus among as many as 18 papers on the desirability of this model. JLIS.it vol. 14,no.  The model is proposed as follows: Papers are evaluated in an ongoing fashion after publication by means of reviews and ratings. Reviews are mini-publications and can be signed or anonymous. Signed reviews and signed ratings both contribute to a scientist's visibility. More important papers are more deeply evaluated as they will receive more evaluations. Scientists are more motivated to perform reviews, because it helps build their reputation (Kriegeskorte 2012).
I argue that the ex post review cannot work and offer an economic argument. Why should researchers feel committed to spend their time in refeering papers? I suggest that the behavior of researchers is influenced by a mix of motivations. Under the ex ante review process -the referee is invited by a person (chief editor, associate editor) -editors are recognized scholars in the community -the invitation to review a paper is usually associated to a motivation (admittedly sometimes standardized) that makes reference to the domain of expertise of the invitee -the invitation is concrete and associated to a deadline, implying that if the invitee does not reply she will be substituted in order to smooth the overall process-in other words, inaction is (gently) sanctioned -invitees know that the number of referees is small (usually two, less often three or four) -editors are committed to make decisions that do not deviate from the suggestions of referees, unless a motivation is provided. Therefore under an ex ante review process the referee understands she -is recognized as a valuable and expert researcher -is accountable to another person -at the margin has an influence on the decision -may have a small, but non-negligible impact on the overall direction of science. In addition, given that the paper has not yet been published, referees can get early access to reference lists that might be of interest, being created purposefully by authors. In this way referees can maintain the state-of-the-art knowledge of fields in which they have only general expertise, but may not have active research and full coverage of the literature. My understanding is that this is another important factor. None of these incentives are at place in the ex post review model. In this case: -no one is personally engaged-the call is directed to a wide (but perhaps unidentified) community -the action does not have a precise time deadline -inaction is not sanctioned -the referee has no effect on the publication outcome -the marginal impact of one's effort is negligible. For these reasons the ex post review process is not economically sustainable. It does not meet any sensible motivation of researchers, at least under the reasonable assumption that the time budget is relatively fixed and the agenda is busy. As a matter of fact there are few working examples of ex post review in scientific communities. However, the discussion on the sustainability of the models associated to Open access com-JLIS.it vol. 14, no. 1 (January 2023) ISSN: 2038-1026 online Open access article licensed under CC-BY DOI: 10.36253/jlis.it-511 pletely ignored these argument. Why is this the case? My answer is that the role of ex post review system in the early radical proposal of Open access was simple but crucial: it eliminates the economic problem of who is paying for the editorial work. The reflection on how the creation of a system of Open science in which all publications are published freely has ignored a thorough economic analysis of the transition problem. Specifically, it has ignored the issue of financing the cost of editorial work. As a result, the diffusion of Open access has created, somewhat paradoxically, a new business model. In exchange for publishing open access, the burden has been shifted from the subscription of journals (hence to universities) to authors. The ultimate source of funding is the same, i.e. government funding, but the channels are deeply different, and so are the incentives. Recent studies examined the structure of the publishing industry after the introduction of the Open access model. Asai (2020) documented the progressive shift of incumbent publishers from the subscription-based model to the hybrid one (subscription + APC), without any erosion in the market power. Maddi and Sapinho (2022) find that the level of APC fees has been growing much more than costs. And Zhang et al. (2022) provided a documented denonciation of the evolution of the publishing industry, showing that the cost of publication and the turnover of incumbent publishers not only were not reduced by the emergence of Open access, but significantly increased. With an interesting effect of heterogenesis of the ends, the original movement of Open access, which had a strong anti-capitalist orientation (Schmitter et al. 2015), actually opened the way to an extreme for-profit model. The issue discussed above is exhacerbated by the emergence of a new publishing model in the last few years that operate exclusively on Open access. The two main actors in the new field are the for-profit companies MDPI, based on Switzerland, and Frontiers. Their business model is based on the publication in Open access and digital format of a large array of scientific journals. These journals charge authors a large fee (APC, Article Processing Charge) in exchange for a fast review. 2 The cost of production do not include the printing and distribution of paper journals. The model is highly attractive for junior researchers insofar as it accelerates their publication rate and contributes to the creation of the early publication record. This business model is creating additional pressure on the peer review system. First, when articles are submitted, there is an automatic call for a large number of referees, selected according to inhouse analyses of domain similarities based on text mining. Knowing that the rate of acceptance of such invitations is low, the editorial management system "fires" an invitation to several dozen researchers simultaneously. New journals are created on the basis of a business model, as niches of publishing markets, not on the basis of the bottom up creation of scientific communities around common scientific interests. Consequently editors are not necessarily respected authorities in the field and do not know personally potential referees. The entire system of trust-based relations between editors and referees, which is at the core of the traditional journal mechanism (Horbach and Halffman 2020), is substituted by machine-based procedures. This is an additional source of JLIS.it vol. 14, no. 1 (January 2023) ISSN: 2038-1026 online Open access article licensed under CC-BY DOI: 10.36253/jlis.it-511 pressure on researchers, who may receive massive requests. Second, in order to ensure the timely review, the management systems of these journals put a strong emphasis on the time of delivery of the reports. Researchers are pushed hard to deliver on time, with several rounds of recall. This is another source of pressure. Recent studies have argued that this editorial model creates a severe risk of degradation of the quality of the peer review process. The median duration of the peer review process is considered unrealistic for a thorough examination of the merit of papers. Oviedo-García (2021) 3 has documented that journals in the MDPI family have a median duration of the complete peer review process (from submission to acceptance) that are significantly and largely lower than common practice of scientific communities. With respect to Frontiers, on the other hand, there has been a serious controversy after the publication in Scientometrics of a paper in which their journals were considered predatory, following the early classification of the Beall' list (Macháček and Srholec 2021). Interestingly, the paper was retracted. Although the inclusion in the list of predatory journals is surely controversial as a whole, the retracted paper raised the attention to the questionable referee practices of new Open access journals. Paradoxically, for-profit publishers have been ready to see that the goal of Open access might be exploited by completely reversing the publishing model and disrupting the traditional editorial workflow. I have seen no reflection on this unintended outcome as part of the analysis of the Open access movement. Zhang et al. (2022) provided a detailed history of the political movement underlying Open access, in which it appears that a thourough analysis of the potential unintended consequences of the APC model was entirely missing.

A modest proposal
Neither the provision of monetary incentives nor the shift to ex post review will work. Other solutions should be explored. My proposal is as follows. The starting point is that publishers should accept to pay the referee work. The current system is unbalanced. Governments pay the salary of researchers, who provide unpaid work to journals. Journals sell the editorial work to governments in two ways: with subscription, under the traditional editorial model, and with publication fees, under the Open access model. In both ways governments pay publishers twice. This model might be sustainable if publishers would charge the editorial work at competitive prices. In this case it would be indifferent for governments who is in charge of the editorial work, whether publishers or other entities (e.g. universities in-house publishing houses). Under conditions of economies of scale and scope it might be shown that large publishing houses are more efficient and a certain level of concentration of the publishing industry would be justified. The crucial point is that we do not have any proof of the economic efficiency of publishers, either in the traditional model or, even less, in the for-profit Open access model. Interestingly, most publishers JLIS.it vol. 14, no. 1 (January 2023) ISSN: 2038-1026 online Open access article licensed under CC-BY  are not subject to the publication of their balance sheet according to separate lines of business and do not publish any independent audit of their accounts. Consequently it is very difficult to examine their average and marginal costs, or their markups. A reasonable assumption is that, given the peculiar type of competition and the rigid demand elasticity, they are close to a monopolistic level of price (Rose-Wiles 2011; Larivière et al. 2015). A recent extremely detailed cost estimate reaches the following conclusions with respect to the subscription model: Taking  Even adding to unit costs a high level of profit margins (i.e. 30% of US$ 4,000, or US$ 1,200) there is still a huge margin of non publishing costs, at US$ 2,200. This margin is a clear indicator of monopolistic power. One might expect that the Open access model, in its battle against the subscription model, has eliminated this monopolistic rent. This is not actually true. According to the authors: The reported average APCs charged by the minority of journals with such fees vary between US$1,400-2,200 depending on the sample. The large difference between these values and even our most expensive cost estimate is at least partly consistent with our hypothesis that the quasi-monopolistic situation of the publishers, due to the non-substitutability of their goods and services, allows them to adopt a value-based pricing strategy also in the APC-OA case, similar to subscription pricing (Grossman and Brembs 2019).
Summing up, both subscription and APC models exploit the rigidity of demand for scientific articles in order to extract monopolistic rents. This creates an increasing level of perception of unfairness in scientific communities, that has nothing to do with an ideological anti-capitalist position. People who accept the working of market economy in other areas are uneasy with the overall economic arrangement of scientific publishing. It is well known that the main goal of the Open access movement is the disruption of the subscription model with the creation of a fully public digital infrastructure (van Noorden 2013; Schimmer et al. 2015;Coalition 2018). It is not clear how to manage the transition, or the migration of large number of editors and referees of existing journals (with their own historical reputation and scientific history) to a unique large digital infrastructure. In any case, the disruption of the subscription model and the transition to a full Open access model will not solve the issue of the strike of referees. Unless the ex ante review is eliminated, as in the ex post review model, the issue remains. Consequently I find the proposal of letting publishers pay the referee work economically sound. There is large economic room in the profits of existing publishers to allocate these expenses. It is not surprising, of course, that publishers are actively tryng to find forms of reward to referees that do not involve monetary expenses, such as credits, free subscriptions, or prizes (Fox and Petchey 2010;Nature 2014;Gasparyan 2015). Another interesting argument against the proposal of paying referees as been offered in short correspondence with Lancet: JLIS.it vol. 14,no.  Paying for rejections will incur additional publishing costs and paying solely for acceptance will benefit only reviewers whose comments are always positive. Finally, a paid peer review would distort the selection criteria of peer reviewers, contributing to the emergence of new commercial peer review agencies, cronyism or nepotism reviewing activities, or specialised agencies to provide reviews on demand, similar to paper mills or agencies that write and fabricate data from scratch (Moustafa 2022). In my view, monetary rewards should be considered. But with a major difference with respect to existing proposals. I do not believe individuals should receive money for the referee work. It must remain an activity that researchers undertake on the basis of the array of non-monetary motivations illustrated above. These motivations should remain intact. At the same time, researchers are permanently worried about the lack of funding, particularly in some areas of research and some countries. My proposal is that publishers pay not individual researchers but groups of researchers. Publishers accept the notion that the referee work can be carried out by a group of referees, not only by individuals. Groups are created bottom up by researchers in similar domains of science. Scientific societies might take active initiative for fostering bottm up initiatives of creation of several small groups of referees. Ethical codes and Guidelines might me developed in order to manage the internal processes of allocation of responsibilities and accountability. The expertise of these groups are carefully delineated through automatic procedures of Topic detection and modeling based on records of publications (including PhD dissertations for junior scholars). Groups should be created on scientific grounds, in order to avoid the "paper mill" problem raised by Moustafa (2022). Given the small size of the group it will be easier to enforce rules for self-monitoring opportunistic behavior. If researchers put less effort, they will be immediately detected by colleagues. The economic analysis of production of public goods has extensively documented the benefits coming from group monitoring and enforcement of prosocial behavior. The group might be held responsible for the timely management of referee work. It might maintain anonimity with respect to authors. Groups are formed around the definition of an entity that will be the beneficiary of payment from publishers, in proportion to the volume of referee work undertaken. This entity might be any administrative unit (university, department) or any recognized sub-units (research team, laboratory). Members of the group will explicitly refuse any provision of allocation of monetary prizes to individuals, or, in other words, they commit irreversibly to allocate the entire amount of money to the selected entity. The money should be used exclusively for research purposes, or for socially valuable goals. As an example, money might be channeled to a collective facility to support young scholars, or to address the gender gap in science, or other valuable goals. In this way publishers would be payed only once, not twice. Referee work would not be unpaid, but would maintain the collective orientation that is a deep value of modern science. JLIS.it vol. 14, no. 1 (January 2023) ISSN: 2038-1026 online Open access article licensed under CC-BY DOI: 10.36253/jlis.it-511