This is an open access article distributed under the terms of the Creative Commons Attribution License (In our roles as Ombudspersons for the journal, the editor asked us to independently review the articles that were published in PHAIR in the years 2022 and 2023 (Bryant & Hancox, 2022; Cameron et al., 2023; Crawshaw & Piazza, 2022; Deckha, 2023; Feltz et al., 2023; Leach & Dhont, 2023; Neldner & Wilks, 2022; Rosenfeld, 2023). Three of these were reviews, whereas the other five presented new empirical data.
Our task was explained to us as follows by the Editor (Hopwood, personal communication May 10, 2024): “the goal is to produce a brief article [...] that speaks to the degree that articles published in PHAIR express a pro-animal position that goes significantly beyond the data presented or in which significant blind spots (e.g., editorial bias or selective presentation of results that favor a pro-animal position) could be identified. To be clear, PHAIR is an explicitly pro-animal outlet; the mission of the PHAIR society is to provide a forum for scientific scholarship that supports justice for non-human and human animals (this is analogous to a journal that focuses on schizophrenia, in which the assumption is that schizophrenia is undesirable for individuals and society and the scientific community should be doing something to prevent and treat it). In that sense, you should expect all published papers to have an explicit pro-animal justice position. The question we would like your committee to speak to is whether this position strays from scientific integrity and misrepresents evidence.”
We reviewed the articles with this goal in mind and did not find much evidence for strong or persistent bias in them. Below, we draw attention to a few potential problems, but we have no reason to assume that these are untypical as compared to other areas of psychology.
One has to acknowledge that there is a fundamental difference between (a) presenting and interpreting new data, (b) reviewing (parts of) the existing literature, and (c) attempting to make a conceptual point by way of argument. These three types of contributions are not mutually exclusive—a given paper may easily have more than one of these characteristics.
The Ombudspersons’ task is the easiest with regard to the first type of contribution because, in these cases the conclusions drawn by the authors may be directly evaluated in light of the evidence the authors present.
For the empirical papers that we reviewed, we did flag some value-laden word choices (“murder”, “exploitation”, “speciesist” and “speciecism”). We flagged these because their use seems to be at least partly reflect the authors’ moral evaluations of certain facts. To be clear, we do not think that it is wrong for scientists to acknowledge their own moral stances. In fact, doing so may even be desirable to enable a more honest scientific debate, especially when the issues are as emotionally charged as the ones at hand here (fairness between humans and other animals).
However, authors should be reminded of the crucial distinction between (a) descriptions of presumed facts (e.g., people’s behaviors and their consequences), (b) the ethical evaluations that many, or some, people have of those facts (e.g., whether their own behavior is morally defensible), and (c) researchers’ personal evaluations of those facts. This distinction should be made as visible as possible in articles published in the journal. Perhaps each article could include a brief section where the authors express their own attitudes toward the article’s subject matter. The rest of the article should then be written in as neutral a way as possible (e.g., “X seems to be the actual situation, and Y is how that situation is evaluated by group Z”).
We also found some instances of what we perceived as overclaiming, given the respective evidence base. For example, Crawshaw and Piazza (2022) examined livestock farmers’ attachment to farmed animals and the farmers’ attitudes about meat; the authors interpreted their findings to suggest that the farmers were engaging in motivated reasoning in order to be able to continue to raise animals and then kill them. However, because the study involved the administration of questionnaire measures completed concurrently, the data offered relatively few constraints on potential interpretations of the results. An alternative interpretation of the same findings would be that farmers just form stronger attachments with their companion animals, as compared to their farmed animals. It is of course possible that some intrapsychic mechanism (“denial”) is used to keep potential dissonance at bay, but that interpretation is not supported specifically by the data.
We therefore recommend that the journal reminds its reviewers that the viability of scientific interpretations hinges on methodology. Sufficiently critical reviewers should point out weak links between data and conclusions, and either recommend rejection of the respective papers, or ask for a more complete discussion of the possible range of interpretations of the data. Making peer-reviews open and citable (by assigning dois) would be a possible way of enhancing reviewer accountability.
Regarding the second and third types of contributions (i.e., reviewing the literature and making a conceptual point by way of argument), the Ombudspersons’ task is more difficult, for several reasons. One may only competently evaluate the biasedness of a literature review if one has a good overview of that literature. Only then will it become possible to judge whether the selection and/or the presentation of the reviewed articles in a given paper is biased. Clearly, this is not feasible in a comprehensive fashion if reviewers are not experts in the respective field.
We did, however, cursorily trace some of the references in the eight papers and did, in fact, detect some evidence for bias in citation behavior. For example, the Rosenfeld (2023) paper cites three sources underpinning the potential benefits of a meat-free diet for athletic and sexual function. However, on closer inspection, these were citations to two studies with a cross-sectional design and a film documentary, none of which support the presence of a causal link. Throughout most of the paper, the author cautiously speaks of potential benefits, but in the “future directions” section of the non-technical supplement, the recommendation is simply “to inform men about the benefits of plant-based diets for their athletic and sexual performance” (p. 2).
We acknowledge that the issues discussed above are universal problems in science, not just in the field at hand. Nevertheless, we recommend that the journal reminds authors and reviewers of the importance of questioning the strength of the evidence-base for strong claims that are underpinned by citations to external sources. The more such claims align with the authors’ own attitudes, the more such scepticism is warranted.
We would like to close with two more general recommendations, in the spirit of fostering the development of this new journal:
First, given the journal’s claim of being interdisciplinary, we think it would be better to have a group of Ombudspersons from different fields; all five members of the current group consider themselves to be personality / social psychologists. Relevant fields that come to mind are Nutrition Science, Ecology, Law, Philosophy, Medicine, and Agriculture.
Second, when reviewing the eight papers, it quickly became obvious how complex the subject matter is, and that researching it will inevitably involve value-related concepts. Any position “pro” or “against” someone or something can never be entirely rooted in scientific evidence. Rather, any such position will necessarily be tied to an endorsement of certain values. To enable a more efficient scientific debate, we recommend that the editor invites an article (or even a series of articles) by experts knowledgeable in conceptual analysis. The goal in this would be to illuminate the relevant conceptual landscape: Which are the most important actors, factual assumptions, and value orientations in this field? What terms should typically be used when referring to them? What are the key philosophical positions that argue for or against valuing human animal life over non-human animal life? To give just one example: what are the main considerations underlying the distinction between human and non-human animals?
Establishing such fundamentals would make it easier to compare individual papers with one another, and it would also make it easier to judge the extent to which a given paper exhibits “bias” (e.g., by ignoring a concept or a value that is commonly accepted as something that should not be ignored). It would also help increase the efficiency of research in this field, because the potential for (motivated) misunderstandings and omissions would be reduced.