News & Insights
Reflections on Impact Evaluation
by KJ Zunigha, impact evaluation officer, Global Partnerships
Last month, I joined nearly 100 of my monitoring and evaluation peers in Washington, DC to talk impact, measurement and data at the 2015 Metrics Conference hosted by the Aspen Network of Development Entrepreneurs (ANDE). Held annually since 2010, the conference provides people working in the impact evaluation space the opportunity to dig deep into the exciting, challenging and often nebulous nature of measuring how much and what kind of impact the social enterprises in which we invest are having on the lives of the people they serve.
Three of the top themes that emerged from the conference were:
1. Right-sizing evaluations
Although randomized controlled trials (time- and resource-intensive studies) are considered the gold standard in evaluation, they are not always the right type of evaluation to conduct. As an impact investor working to categorize and evaluate the impact we and our partners have at the household level, Global Partnerships (GP) needs to look at the size of the project, the question(s) we want answered, and the resources we and our partners have available to help guide evaluation design. We must always keep our partners and their clients at the forefront of this process so that any evaluation activities we undertake do not overly burden or interrupt the work of the people we are ultimately working to serve. The right evaluation must be as rigorous as possible within the bounds of the environment in which we and our partners operate.
2. Understanding different audiences and communicating effectively
Impact evaluators collect, analyze, synthesize and report impact data for a wide variety of audiences, each of which has different needs. Some may need only an "elevator pitch" or talking points, some want to dive deep into the details, while others are looking for a specific kind of finished product—a case study, a story, a trend report or projection. There are countless ways we can measure, discuss and share data, so as evaluators it is critical for us to understand the needs of our various audiences so that we are able to communicate our work effectively.
3. Maintaining continuity with investors at all stages
GP provides debt financing to social enterprises that are looking to grow and scale their organizations, and may be one of a number of investors and supporters. Too often in this sector, investors at various stages in a social enterprise's development don't communicate their measurement approaches. This creates the likelihood of a dramatic shift in measurement activities and data requirements—and a potentially significant burden for the social enterprise—as it grows and seeks new types of capital.
While measurement approaches and indicators may change as a social enterprise grows and matures, it is important that all investors at every stage of a social enterprise's development have visibility into each other's measurement activities to create a more streamlined approach and a sense of continuity.
Tara Murphy Forde, GP's Director of Impact & Strategic Initiatives, led a conference session on this topic with Vox Capital's Rebecca Obara, looking at the impact frameworks of their respective organizations. The session focused on Vox's custom, hands-on approach to working with its investees as an early-stage investor, and GP's approach that focuses on impact at the household level across our portfolio and multiple initiatives. One way GP supports visibility into our impact framework is by feeding our data into IRIS (Impact Reporting and Investment Standards), a database of social impact metrics intended to help create a common framework for impact measurement.
Perhaps the greatest overall takeaway from this year's ANDE metrics conference is that, with our broad portfolio of initiatives, Global Partnerships' best opportunity to gain meaningful data and insight is to approach impact measurement with flexibility. With an eye on the big picture, the data we collect from each of our partners can tell us about what their work really means for the people they serve.
If you enjoyed this post, you might also like:
- Meet KJ, GP's new impact evaluation officer
- 3 emerging trends in impact evaluation
- Why is measuring impact so complex?