Should we replace the traditional ways of evaluating scientific quality with indices of mentorship?
A new paper in PLoS Biology (click on screenshot below) calls for a thorough revamping of the way scientists are evaluated for the quality of their work, replacing traditional methods of assessment (research productivity and quality) with evaluations of mentorship. The reason the authors want to dismantle the traditional meritocratic methods of evaluation (based, they say, but erroneously, on citation rates) is that these methods are biased against women and minorities. But their evidence for that is almost nonexistent, and, in the end, what they are doing is replacing the traditional empirical purpose of scienceto understand the universewith a social purpose: to increase social justice. I want to emphasize at the outset that insofar as science is racist, with the racism built into the system, that needs to be changed. The authors feel that science is deeply racist, with the result that women and minorities dont get cited, dont get grants, dont get tenure, and in general achieve less than white people or men. If that is due to current practices in science, or current biases of scientists, then this must change. But the evidence that structural racism is pervasive in science today is nonexistent. (Some scientists, of course, are racists, but the system itself, I claim, is now set up to favor women and minorities, not hold them down.) Further, as Ive said before, we do need a form of affirmative action in education and in science. It simply wont do if the present system somehow results in a glaring deficit of women and people of color in science. I think more equitable representation (though not necessarily proportional representation) is a moral imperative, if for no other reason than to provide a form of reparations for groups who were held back years ago and havent yet caught up. What I do claim is that, at present, science is not nearly as racist as the authors represent. In fact, if youre involved in American academic science, you know that departments are scrambling hard to get minority and women faculty and graduate students. The reason we have trouble getting minorities, however, is the pipeline problem, based on a system of oppression and cultural differences that traces back centuries, and must be rectified not by changing the criteria for advancing in science, or sniffing out a systemic racism in science doesnt exist, but by allowing everyone to have an equal opportunity to become a scientist. And that involves big societal changes that afford equal opportunity from birth. Ive said all this before, so let me present the authors abstract, which pretty much tells their tale:  Success and impact metrics in science are based on a system that perpetuates sexist and racist rewards by prioritizing citations and impact factors. These metrics are flawed and biased against already marginalized groups and fail to accurately capture the breadth of individuals meaningful scientific impacts. We advocate shifting this outdated value system to advance science through principles of justice, equity, diversity, and inclusion. We outline pathways for a paradigm shift in scientific values based on multidimensional mentorship and promoting mentee well-being. These actions will require collective efforts supported by academic leaders and administrators to drive essential systemic change. In short, the authors want to replace citations (who cites your papers in their papers, and how often) and impact factors (the quality of the journals in which you publish) with multidimensional and holistic mentorshipa mentorship not designed to produce scientists who find out things, but who are healthy, happy, mentally stable, dedicated to equity, and representing all ethnic groups. This is a proposal to make science into a vehicle for social engineering. Heres a figure in which the authors purport to show the problem. Note that their characterization of how science is done now is said to focus exclusively on citations as a measure of ones impact on the field (encompassed by the H index, which didnt exist for most of my career), while the inclusive view focuses on basically everything. Much of the PLOS paper focuses on the inequalities said to be faced by both women and minorities: fewer citations, fewer awards, less grant funding, greater difficulties in publication, and so on. If one evaluates people based on these factors, especially citations, one is said to be exercising biases that hold down women and minority representations. But I know of no academic vetting process that just counts citations and ignores mentorship as a way of evaluating quality. What is important is not just impact measured by citations, but by the quality of ones work, assessed in many different ways. The most important is reading your papers, seeing what youve found out, and learning whether other scientists express interest in your work. In the promotion and tenure committees Ive been on, citation counts have not ever been mentioned. Rather, a candidates papers are read and discussed, letters solicited from people in the field are considered, production of graduate students is noted, teaching is assessed (not so heavily at a research school), andin some places, but not Chicagooutside research support is assessed. Citation numbers are only a small part of this process, and at Chicago werent even considered when I was on the promotion and tenure committee in biology. The evidence is thin. The authors cite older papers in support of this thesis, but completely ignore newer papers showing that virtually every metric of womens advancement in science is now on par with men, that there is no perceptible bias against either women or minorities in assessing grant proposals (nobody ever cites this important paper because it doesnt meet what people expected to find), that the lower rate of funding by black scientists isnt due to biases against them, but to differential choice of fields having different amounts of funding, and a survey of faculty and student hiring preferences showed that minority candidates and women are preferred above white candidates and males, especially by faculty. Removing not just citations but other ways of evaluating scientific productivityall aimed at answering the question, Has this person pushed our knowledge forward?is a surefire way to erode the quality of science. As I said, our main aim is to find out stuff, not act as a vehicle for social justice, though we should of course behave towards our students and colleagues in an unbiased fashion. This itself involved problems, because, at least for academic mentorship, exposure to science means exposure to research. One of the signs of a good mentor, like Dick Lewontin, is that they produce lots of students who themselves produce lots of research, so their reach is extended. First, having research experience always helps you get a job in industry. As for teaching, I would argue that exposure to research, even if it isnt published, is an essential part of producing someone whos a good teacher of science. Good teachers understand how research is done, and you need to learn to do that by doing it, not by reading about it. In the end, research should always be part of any scientific training. And if youre at a teaching school, or teaching science in secondary school, citations and research are of minor or no importance. But Davies et al. arent interested in this kind of mentorship. They want a holistic mentorship in which research is downplayed in favor of producing students who conform to social-justice expectations. These students are mentally healthy as well as having bystander intervention training and anti-bullying and antiracist mentoring and teaching practices. Ones mentorship must promote justice, equity, diversity, and inclusion in science. Other practices of good mentors include these:  To ensure that training opportunities become valued by participants, institutions may consider implementing mandatory participation by requiring training for career advancement or as prerequisites for recruiting mentees. However, training programs should be mindfully designed to engage those who may complete training for inauthentic reasons. [JAC: What???] Discussions of topics covered in training should become standard practice at regular events including faculty meetings and retreats and graduate student association meetings. Undergraduate programs can include discussions of unconscious bias and how such biases influence classroom dynamics. And we must become experts in mental health as well:  While good mentorship can foster a sense of belonging in science for the mentee, relationships of many mentees from marginalized groups with their mentorswho are often from the majority groupare not always positive, leading to health issues, such as insomnia and anxiety, and lower retention of these groups in science (reviewed in [93,104]). In order to effectively mentor, all mentorsparticularly those who are not familiar with the experiences and perspectives of systemically marginalized scholarsshould engage with cultures, communities, and perspectives that differ from their own, connect with communities that are working toward creating justice, equity, diversity, and inclusion, and support institutional change already underway. In addition, increasing representation from marginalized communities throughout institutional hierarchies provides greater opportunities for mentees to find mentors with which to build meaningful relationships. Of particular concern is the recently highlighted decline in mental health of many academics and a growing crisis at the graduate level. Graduate students are at least twice as likely to experience mental health challenges, such as anxiety and depression, compared to the general population with equivalent education. This trend is even more striking for women of color in STEMM, who are facing systemic sexism and racism, along with daily microaggressions and safety concerns. Sexual minorities and LGBTQ+-identifying people are also subject to discrimination that adversely affects their well-being, mental health and, ultimately, retention in STEMM fields. Laboratory work, field work, and simple existence in the academy can often place marginalized groups, including those with disabilities, at risk of injury, harassment, bullying, and assault). To combat these challenges, specific strategies for safety and well-being must be supported at the research group, departmental, institutional, and funding organization levels. Multidimensional mentorship clearly requires expertise in psychotherapy. While good mentors are sensitive to their students psychological needs, there are other groups within universities designed deal with these issues. And emphasis on all of them dilutes the very purpose why one does science, turning it into a vehicle for promoting social justice in the community. But thats exactly what the authors want. I am drawing to a close, for Ive seen paper after paper like this in the last few weeks, all suggesting that we ditch traditional ways of assessing scientific merit, because those ways give a disadvantage to minorities and diminish social justice. I would suggest, though, that there are better vehicles than science for promoting social justice, and that we shouldnt turn our field into an ideological juggernaut. Of course we must provide equal opportunities for all, but I think academic science is doing a pretty good job at that. There will surely be dissenters touting their own lived experience of oppression, but the data we have now suggests that science is doing a pretty good job at promoting equal opportunity. And I have not yet heard a way to improve the mission of scienceto find out truththan the ways were doing it now. Were casting wider nets for talent, and fighting hard to eliminate any biases that we can find. All I can say is that I disagree with the idea of replacing the criteria currently used to evaluate someone as a scientist with criteria mainly concerned with mentoring people in a social-justice-y way.