Structural Racism Exists in Evaluation
In the months since the murder of George Floyd (and Breonna Taylor, and Rashard Brooks, and Ahmaud Arbery… https://saytheirnames.io/), the widespread outrage and large-scale protests resulted in a long-overdue awakening for many white people, including myself. Like many others, I have begun the process of educating myself on my role in the systemic racism and white supremacy that exist in this country, and my responsibility to push back against these systems. In the face of a national tragedy so devastating, so extensive and going back so many generations, the question of “What Can I Do?” is intimidating to say the least. Although there are many possible answers to this question, one common theme that appears often is to start by looking inward, at the privileges and biases that we carry around every day. Whether conscious or unconscious, visible or invisible, they impact our lives, and the lives of others, in many ways; but in this post I want to focus on how they impact our work as evaluators.
We make many decisions, big and small, in designing and conducting an evaluation. From choosing which stakeholders will have input in identifying evaluation questions, to selecting or developing data collection instruments, analyzing and interpreting the data, and finally reporting findings. At every step of this process we should question how our decisions are being influenced by biases, white privilege, and understanding (or lack thereof) of the culture(s) and race(s) of the target population. Not whether our decisions are being influenced, but how. This is not a simple or quick task. As seasoned evaluators it may be difficult to objectively question the theories and processes that we have relied on for years. We will never be free from our biases, but awareness is the first step toward reducing the likelihood that our evaluations will unintentionally cause harm and/or perpetuate racist policies and systems.
The good news is that there is a wealth of information to help guide us in the process of centering race and culture in our work as evaluators. I have only just begun my journey but I wanted to share some early takeaways that have been eye opening for me, along with relevant resources:
Recognize that “Evaluations cannot be culture free” (AEA, 2011): The American Evaluation Association (AEA) published a Statement on Cultural Competence in Evaluation in 2011, which was developed over the course of six years with contributions from a wide range of evaluators and experts in cultural competence. It provides a thorough overview of why evaluations can never be free from cultural influence and why “cultural competence is an ethical imperative” in evaluation. A Child Trends working paper titled How to Embed a Racial and Ethnic Equity Perspective in Research (2019) takes a more in-depth look at every step of the research process, providing a detailed set of guiding principles for researchers and evaluators to “intentionally embed a racial and ethnic equity perspective” in their work. These resources ask us to recognize how culture influences aspects of evaluation that may appear to be neutral – such as the development and testing of standardized data collection tools, and the definition of social problems and what constitutes success – and to be careful not to hold the white experience as “the norm to which all populations should aspire” (How to Embed a Racial and Ethical Equity Perspective in Research, 2019).
Reconsider widely-accepted language that may be problematic: We all know that language is a powerful tool, yet it’s easy to assume that certain terms are “standard” or “neutral” when in fact someone from a different perspective may disagree on their meaning or implication. After reading through A Guide to Coded Language in Education Vol I & II (2020), I was shocked at how many educational terms appeared on this list. I had never taken the time to think critically about how language such as at-risk youth, achievement gap, and college readiness centers whiteness and engages in victim blaming.
Gain a better understanding of your implicit biases: There are many excellent books to help with individual journeys to uncover personal biases – “How to Be an Anti-Racist”, “White Fragility”, “Me and White Supremacy”, to name a few. If you need help to stay on track with changes you want to make, sign up for America & Moore’s 21-Day Racial Equity Habit Building Challenge. You can also take an Implicit Association Test (IAT) through Project Implicit, a non-profit collaboration of researchers from different universities. These quick tests are available on a range of topics including race, religion, age and disability; the result will reveal whether you have an automatic preference for a certain group within each category, and the strength of your preference, if any.
Read what other evaluators are saying on this topic: The University of Illinois’ Center for Culturally Responsive Evaluation and Assessment, and the Equitable Evaluation Initiative are two groups dedicated to embedding equitable practices within evaluation, and both offer various resources and publications on this topic. The AEA365 blog published a series of posts on white privilege: So We’ve Got Privilege! What Are We Willing to Let Go of?. One of these posts, Letting Go of White Saviorism, includes a simple but powerful table showing “how white saviorism distorts reality in the evaluation field.”
Continually ask for feedback from clients and program recipients: At MI, we believe that working in collaboration with our clients at every step is the key to a successful evaluation process. To ensure this is an equitable process, we must commit to transparent discussions about race, culture and power dynamics, with clients and program recipients. Evaluators can also use this self-assessment tool, Is My Evaluation Practice Culturally Responsive?, which is designed to be taken “at regular intervals to identify changes in areas of strength, as well as targets for growth.”
The final important take away is to recognize that this is an ongoing process that we must work at every day; cultural competency is not something that can be achieved. Becoming anti-racist is not something that can be achieved. The AEA Board of Directors published an additional Statement on Systemic Racism in 2020, that summarizes the two levels of work to be done:
We call on evaluators to do the individual-level learning that will help them recognize the ways that evaluation can cause harm. It is also our collective responsibility to step up and find ways to use our unique skills to dismantle racism and systems of oppression while creating healing and safe spaces to build bridges to a more equitable, democratic, and just future (AEA, 2020).
It is encouraging to recognize that evaluation has the ability to affect larger change, and I hope one day to move my focus from a personal goal of conducting equitable evaluations, to a societal goal of using evaluation to help create anti-racist policies and systems. But for now, I will begin with me.
Nora Phelan is a Research Associate at Measurement Incorporated. Please visit this website for a description of our programs and services.
A special thank you to Jessica Coles, Culturally Responsive Specialist with the Capital District Regional Partnership Center, for recommending some of the resources used in this post.