Measurement is an important tool in helping us to assess the progress of a strategy; it is equally important in helping us rearrange power dynamics and recognize our privilege. Because we use measurement for learning, it inherently holds us accountable for the changes we seek and the degree to which we are living our values. Measurement and accountability proceed arm-in-arm as they are needed for us to make the most informed decisions.
We do not develop the measures in a vacuum but customize our metrics to a strategy. Our process is inclusive, using feedback loops to include diverse perspectives. We intentionally call on the expertise of our grantees, internal and external advisors, Board, and evaluation and learning partners to select measures that are reliable and realistic.
To accomplish this, we ask ourselves several questions in the process of deciding what to measure, such as:
- Who, what, and where do we want change?
- When do we expect change to occur?
- What is the vantage point (e.g., systems, policies)?
- What are the contextual and cultural factors at play?
- What types of data yield the most useful information?
Each evaluation has outcome, impact, and contextual measures. Outcome measures are used for information on the work on-the-ground and close to our funding and support. Impact measures inform us about the ultimate contributions of our work. We know there are external factors that can hinder progress, so we select a handful of measures that inform us about the context we are working in. We follow each from baseline to the end of a strategy, though some may change based on evolving circumstances.
Since 2015, in our
Criminal Justice evaluation, for example, measurement has been used to determine progress towards reducing jail populations, creating a fair and just use of jails, perception about jails, and increasing the number of jurisdictions undertaking reform.
Outcome measures have given us information on jail populations: admissions, racial and ethnic disparities, length of stay, and flow into prisons. We compare grantee sites to themselves overtime, to each other, and to similar jurisdictions overtime.
For impact measures, we are following national jail trends, the national conversation about jails, and how criminal justice reform is spreading across the country. We use a variety of methods—such as polling, media scans, and surveys—to determine if implemented reforms are reducing jail populations, particularly for overrepresented populations, like Black, Native Americans, and Brown people.
And we follow contextual measures, such as public safety and county expenditures, to give us the information we need to identify the consequence of changes to criminal justice systems at our sites and nationally.
We know that the measures selected will not be perfect and have limitations, so we revisit them periodically to determine if it is providing us useful and actionable information. This increases the likelihood that the decisions we reach are well informed and align with our values and goals.
