Numbers Count

Why measuring and tracking data through our major survey matters

by Justin Boles

Some of the words we bat around today are as ubiquitous as the air we breathe. Data is one such term. Like air, we don’t think much about data until we need it and don’t have it. Data surrounds us and powers so much of the activity we see in modern life, so in its absence, we’re deprived of something that helps us move forward with force and intentionality.

Ask several people to define data and you’ll get just as many definitions. Even its pronunciation and plurality are in question. (We at Citygate Network prefer the pronunciation that rhymes with beta, and we’ll often ignore the plural form of data—datum—because it’s an odd word.) The term was born from the Latin word dare, or to give. Data, then, is what’s given. Given means, “what is beyond argument,” and that’s what makes data so valuable.

People are prone to seeing the information they wish was there instead of what truly is there; that’s called “optimism bias.” I’ve been part of decisions made with optimism bias, and those usually turned out to be learning opportunities, so to speak. We also tend to favor information that confirms the beliefs we already have, right or wrong; that’s called “confirmation bias.”

We make many decisions at the subconscious level, and sometimes we do have to go with our gut; but when the Holy Spirit speaks, He overrules everything—data included. But it seems that we live primarily in that middle zone, where God wants us to wrestle, research, and strive to understand. The Message’s version of 2 Timothy 2:15 reads, “Concentrate on doing your best for God, work you won’t be ashamed of, laying out the truth plain and simple.”

In the world of program outcomes, data can demonstrate progress, change, and transformation. Data also helps to diagnose and correct problems in your programs.

Plain and simple. There are few things as plain as numbers, which is why data is usually expressed in numbers. But while numbers are simple, we must be careful to interpret them correctly. You’re probably familiar with the quote by British economist Ronald Coase: “If you torture the data long enough, it will confess to anything.” By cherrypicking data or by framing it in certain ways, we can spin data into information that’s less useful and even destructive. That’s why we must pour much thought into what we measure, why we measure it, how we measure it, and what biases might creep into the process if we’re not safeguarding it. Regardless, data is extremely useful for several different purposes.

1. Data is useful for measuring outcomes.

Collecting and measuring program outcomes data is increasingly important for rescue missions and kindred ministries. For quite some time we’ve advocated that our members measure outcomes in at least four primary domains: relationships (with God and others); health (physical, mental, sobriety); economics; and housing. Citygate Network member Helping Up Mission (HUM) in Baltimore measures 15 outcomes in its addiction recovery program. These outcomes, which staff strive to see realized in the lives of those who go through their program, nest into five major domain areas: committed to sobriety; introduced/reconnected to God; emotional and physical wellness; restored relationships; and re-integrated into the community.

While these areas may look a little different from one ministry to another, define the win for your particular programs. In the words of HUM CEO Bob Gehman, “Outcomes are the set of sails that allows the wind to carry the board toward its destination. They allow the entire crew to be knowledgeable of where they’re going and work together to get there.” In the world of program outcomes, data can demonstrate progress, change, and transformation. Data also helps to diagnose and correct problems in your programs.

One example of how data can improve programs is in how Seattle’s Union Gospel Mission has used it to evaluate and fortify program retention. Charry DeAndres, the mission’s director of data and analytics, explained, “We know that the longer an individual stays in program, the more likely they are to have long-term success in recovery. By measuring attrition throughout the program, we were able to identify the phase when most people left.” In this example, data analysis not only found a correlation between program length and success, but it also pointed to the program phase when clients were most likely to exit.

The mission was then able to drill down even further into its analysis. “We used that data to evaluate why individuals were leaving at that specific point in program, compared to other times,” Charry said. “With that, we were able to rework our recovery and counseling structure in that phase. The data then let us know if the change had the desired effect—and it did!”

It’s important to note that, even when a client exits the program earlier than we’d like, there are still seeds planted that God can cultivate. Bob Emberger, executive director of Whosoever Gospel Mission in Philadelphia, says, “We view each resident on a continuum of progress. His success is not all or nothing. We try to capture a man’s success in achieving all kinds of outcomes—short-term, mid-term, and longterm. A short-term goal would be to achieve 90 days of clean time from drugs; a mid-term outcome would be to secure a full-time job that pays above minimum wage; and a long-term goal would be that this resident is still working fulltime and living independently one year later. Measuring all sorts of outcomes and celebrating small victories helps us to explain our ministry and its impact to our funders and supporters. It can also be a great encouragement and motivation to both our residents and our staff.”

Bob Gehman echoed that point: “Outcomes data allows all who are participating in the journey to measure the difference they’re making. You know when you’re winning and when you’re losing; who is contributing and who is not. This helps create a winning organizational culture.”

2. Data is useful for raising money.

Securing necessary resources is another marker of winning. Imagine you have a huge program need, and it will take big money to meet it. You find a foundation that loves giving to projects just like the one you have in mind. As you begin working on the narrative, words flow like water. You draft a masterful executive summary, a compelling needs statement, and a tight project description with a realistic budget. But then you arrive at the measurable outcomes section.

Make sure you’re prepared—before you ever realize the funding need—to give this section due diligence. Measurable outcomes help secure grant funding, particularly for large gifts, and having outcomes data from existing programs and projects is a wonderful way to demonstrate to future funders that your organization values measurable results.

Nearly 60 percent of our member ministries work with at least five foundations in a typical year. Foundations want meaningful results to come from their grants—not only to highlight grantees’ accomplishments, but also to illustrate their own accountability and appeal to donors. More and more, sophisticated donors also want some of those results to be measurable.

“Institutional, agreed-upon outcomes data is a game-changer for fundraisers,” Bob Gehman explains. “I don’t think HUM, or any other organization for that matter, could get the large, transformational, investment-type gifts without outcomes measurements as a case for support. It just doesn’t make sense to the investor to invest in vagueness.”

Very recently, a potential funder contacted Citygate Network for benchmarking outcomes data. He wanted to know how a particular mission compared to other missions in our network with helping people become sober, procuring stable housing, and earning livable wages. He also asked about output-type data, such as nights of shelter and meals served. While we were able to provide some benchmarking data, we would love to be able to provide a more complete set of metrics to help our members win more grants.

“Outcomes data allows all who are participating in the journey to measure the difference they’re making. You know when you’re winning and when you’re losing; who is contributing and who is not. This helps create a winning organizational culture.”

The M.J. Murdock Charitable Trust has given $24.8 million through 163 grants to rescue missions and similar ministries. Dana Miller, the trust’s senior program director for grants, points out that compelling outcomes usually include both quantitative and qualitative aspects. “When we are working with nonprofits, we do look for quantitative outcomes that can be benchmarked, but we also want to engage organizations around qualitative or anecdotal outcomes as well,” she explained. “For example, a new community center may serve a measurable number of children, but the increased bonds of the community don’t necessarily show up on a scale.”

Organizations such as those in our network are working toward, in a sense, quantifying qualitative results. It’s critical to define the win for people who are served, but missions are also learning to assign values to progress toward that win, not just the ultimate goal. We need a subjective component to this valuation because kingdom work isn’t easily quantifiable, but if we can agree on some broad signs of that progress, we have a benchmark to use to improve programs and raise more money for ministry.

Dana described Murdock Trust’s view this way: “Albert Einstein famously said, ‘Not everything that can be measured matters and not everything that matters can be measured.’ We believe this is true in the nonprofit sector, and we seek to strike a balance in how we evaluate the work of our grantees.”

3. Data is useful for influencing policy.

The relationship between government and faith-based nonprofits has reached a crucial transition point, and the effects of this November’s election on that relationship can’t be determined yet. But we’re learning that the government can indeed be a partner in our work.

Fifty-five percent of our members participate in their area’s Homeless Management Information System (HMIS), which is an information technology system used to collect client-level data and data on housing and services provision. This data feeds government policy and dictates the disbursement of government funds to nonprofit service providers. What’s more, nearly as many of our members have representation on their local Continuums of Care (CoC), and 10 percent of our members actually chair their CoCs. Many participate in HMIS and their CoCs without expectation of—or desire for—funding for their own ministry.

We believe that multiplication is far better than division, so we’ve been working for years to correct misguided government assumptions about our members’ work, as well as to help members see how they can partner with government without compromising core values. Data plays a key role in this endeavor.

For many members, their first foray into federal funding came this year when nearly 80 percent of our members received funds from the Small Business Administration’s Paycheck Protection Program. This seems to illustrate that, when the government is willing to partner without heavily influencing methodology, rescue missions are not opposed to utilizing government funds.

One powerful way that rescue missions and kindred ministries can safely access government grants is by convincing legislators that their programs are effective at solving public problems and even saving tax dollars. A 2017 study initiated by Citygate Network—“Assessing the Faith-Based Response to Homelessness in America” (www.citygatenetwork.org/baylorstudy)— estimates that faith-based organizations create $8.27 in taxpayer savings for every $1 the government invests.

The study, which was conducted in 11 cities where Citygate Network members operate, further shows an estimated $119 million in tax savings in those cities during the three years following implementation of faith-based residential recovery and job readiness programs. More than eight dollars for every dollar invested—that kind of multiplication is a compelling reason for the government to give ministries latitude in methodology.

We believe that multiplication is far better than division, so we’ve been working for years to correct misguided government assumptions about our members’ work, as well as to help members see how they can partner with government without compromising core values. Data plays a key role in this endeavor.

Eighty-eight percent of Citygate Network members feel our government engagement activities strike the right balance. Perhaps this year, more than any other, we’ve seen improvement in government policy and postures toward faithbased organizations. By using data to make persuasive arguments, we’re making progress on Capitol Hill—and you can, too.

The goal for this data isn’t to reduce transformational work into numbers; the goal is to provide waypoints to inform ministry, influence policy, gain public trust, and attract partners.

Speaking of Progress

Some 80 percent of our members report that they do track outcomes data, and nearly 60 percent of our members track data in at least the four domains that we promote. These percentages have increased dramatically over the past few years. The next step is to translate that data into a universal outcomes language and collect it in a central repository.

The goal for this data isn’t to reduce transformational work into numbers; the goal is to provide waypoints to inform ministry, influence policy, gain public trust, and attract partners. Data is certainly not the be all and end all—that honor belongs to Jesus Christ and His work on the cross—but it adds a great deal of value to important conversations. The data conversation will continue within Citygate Network.

Editor’s note: You may notice that data points are sprinkled all throughout this article. Most of these statistics were generated by our recent member survey, and we greatly appreciate your participation. See—we use data, too!

Justin is vice president of Citygate Network, overseeing the programs, products, services, and education that member ministries turn to for support. In the 16 years prior to joining the network, Justin served as a missionary journalist, communications and marketing director of Christian Camp and Conference Association, marketing consultant, and executive director for several other organizations.