Data Management: Gateway or Gatekeeper to Equity and Validity?

We have come (or have been led) to believe that data is neutral, objective, and true. If only the world we live in was that clear and simple.

Data collection is based on what we ask, what is of interest to us, and what we deem to be important and worthy of knowing. This is all determined by who we are, where we sit, what we value, and how we see the world. “Our biases remove our ability to be neutral, and we can’t claim that what we collect is representative of the full truth (or even the most important truth).”

We at the Equitable Evaluation Initiative (EEI) believe that those who are part of shaping philanthropy have a moral imperative to approach their work in ways that contribute to equity. And we believe this is even more essential for those engaged in efforts specifically related to equity. We don’t have a solution, but we do have an invitation and a framework for your consideration that is rooted in three principles. Although we have the word “equity” in our name, we speak to a more accurate and complex understanding and commitment to validity, one that is appropriate for the 21st century endeavors in which many foundations and nonprofits are engaged. We are imagining what might be possible if evaluation was conceptualized, implemented, and utilized in a manner that is consistent with and promotes equity.

Assuming you find logic and merit to the above, we invite you to explore how we can “check” ourselves to ensure that:

1. We seek the most relevant and valid data to deepen our understanding of situations and conditions and to set optimal indicators of progress toward stated aims
2. Analysis and sense making is informed and grounded in context, history, culture, and a variety of other factors, such as perspectives with diverse insights
3. Our pursuit of proof or evidence does not do unintended harm, particularly if our aims are in service of social justice and equity

In the role of a grants manager we invite you to consider how these principles can be applied as you think through everything you do, from how you decide which data management system you purchase, to how it is built out, the types of data it can hold, and the reporting options it offers. Here, we share the three EEI principles and some questions to engage your thinking:

Principle 1  – Evaluation and Evaluative Work Should be in Service of Equity

• Production, consumption, and management of evaluation and evaluative work should hold at its core a responsibility to advance progress towards equity.

A question to ponder as it applies to your work as a grant manager:

What do you know of the way in which the data was collected? In your role, you can help program managers understand if they have sufficiently interrogated the data and its source and validity.

Principle 2  – Evaluative Work Can and Should Answer Critical Questions about the Following

• The ways in which historical and structural decisions have contributed to the condition to be addressed
• Effect of a strategy on different populations
• Effect of a strategy on the underlying systemic drivers of inequity
• Ways in which cultural context is tangled up in both the structural conditions and the change initiative itself.

A question to ponder as it applies to your work as a grant manager:

Is there a way to understand differential effect based on how the data was collected? To what degree did the strategy development attend to systemic drivers of the issue being addressed?

Principle 3  – Evaluative Work Should be Designed and Implemented Commensurate with the Values Underlying Equity Work 

• Multi-culturally valid
• Oriented toward participant ownership

A question to ponder as it applies to your work as a grant manager:

How and in what ways are context, power, and relationship considered in the analysis of the data? What was happening at the point of inquiry or analysis that needs to be taken in to consideration as you evaluate your data?

Evaluation is among the last organizational functions to be examined and revamped in alignment with values or intentions (equity or otherwise). By failing to bring evaluative practice in to alignment Foundations may be unintentionally engaging in inquiry and sense making that is not as insightful and meaningful as need be and, even worse, may be reinforcing norms and practices that are at odds with their strategy and goals.

Grants managers have the ability to ask program managers a different set of questions that can challenge and clarify thinking, which can lead to clarity on the why and what of the inquiry. And IF equity is your thing, how might that inquiry be in service of decolonizing and privileging information and knowledge.

We believe that grant managers play a powerful role in shaping a world where evaluation is conceptualized, implemented, and utilized in a way that promotes equity. If you’re interested in learning more about the ways you can get involved with EEI, please head to https://www.equitableeval.org/. Thoughts? Questions? Share them with us at info@equitableeval.org.

Subscribe to our monthly newsletter for emerging learning and updates on how our work is unfolding: http://bit.ly/eeinewsletter