When the Pursuit of Data Just Isn’t Worth It

Linear Composition 3, by Andrew Mechum

Linear Composition 3, by Andrew Mechum

Data is everywhere and, for many, is everything. It’s put on a pedestal where it is both loved and admired. It’s protected and cared for by smart, devoted people. And, it’s both the question and the answer when we cross paths with the OMB or Congress.

I started to write that federal programs “these days” have become synonymous with massive data collection, analysis and reporting exercises, but we all know that’s not 100 percent true. For as long as we have been a nation, we (the public) have demanded from our government (and our contractors) a thorough accounting for the money spent and the accomplishments achieved. In today’s government, that means data and that’s all good.

The issue? A little taste for good data leads to cravings for more. Without realizing it, we’ve become zombies with insatiable appetites for fleshy spreadsheets. That’s not so good.

Why? Data is expensive—in fact, far more expensive than we like to acknowledge. In government and in contracting, data is the fancy, bubbly, bottled water that we treat like tap.

We rarely talk about the cost of data because we believe data makes us smarter and better organizations. Federal executives want insights and answers—so they ask for more analysis. Program managers with their staffs and consultants want to be responsive and to show their program’s value—so they collect data to power such analysis and start crunching. They might wince a little in the process of getting there, but ultimately they move heaven and earth to produce polished, data-rich reports.

Yet for the dozens of well-purposed, well-intentioned federal programs, such as the Federal Real Property Profile (FRPP), the Employer Information Report, the Energy Review, and federal information technology (IT) investments, data requests like these are not making them smarter or better. They are merely creating a data collection burden on both federal employees and their contractors.

Each of these programs (and the many others like them) has a purpose. And all the data they produce has potential value. But how much did it cost to obtain it? Where is that data now? And who is using it?

The FRPP exemplifies a good idea gone wrong. If you’re not familiar with this program, the FRPP is a skim of a federal agency’s real property data and metrics. Agency staff collect the data and send it to the GSA. Using that submitted data, the GSA then produces an annual summary report of the federal government’s footprint.

And then not much else happens.

Now many important things are happening in real property at each of the agencies that reports data—but the FRPP data at the GSA is not reflective of that activity. In fact, the rules around FRPP data make it too hard to easily snag data from other systems and too superficial to do much interesting, useful analysis with it on its own. Yet agencies still bear the cost of gathering the required data, checking it, and submitting it for no return on their data collection investment—except for a check in the compliance box.

In their book, the “Agile Culture,” Pollyanna Pixton, Paul Gibson, and Niel Nickolaisen advise, “Always ensure that the cost of collecting the metric is significantly less than the value that it can deliver. And we do mean significantly less.” Recognize that the cost of collecting data is not zero and can sometimes be very high, especially if it involves continuous action by the delivery team members. This cost is often ignored and can have a huge negative impact on team productivity.

Even good programs can grow increasingly expensive because we need—and want—to understand broad, complex problems. There is no doubt that our pursuit of data and answers has yielded some insights, avoided some crises and enabled some right choices being made the first time. Yet is it right, or even sustainable, to pursue data without periodically asking ourselves how much it is going to cost to obtain the data?

How do we think the data we seek is going to answer an unanswered question? Is the tradeoff between the expense to collect the data and insight gained worth it? If the answer to this quick “gut check” is yes, then by all means, pursue that data with purpose and seriousness. However, if the answer is less than a resounding “yes” or we’re struggling year after year to ensure collection compliance, maybe it’s time to admit that the data isn’t needed or as valuable as once thought. Seeking data at all costs isn’t a wise (or sustainable from a political or budgetary perspective) approach.

The alternative is to treat data like other investments and to measure its return on investment. Keep collection and refinement efforts in check by continuously weighing the costs and benefits.

Robin Camarote

I'm the co-founder of Federal MicroConsulting and strategic planning consultant based on Falls Church, VA. I am intent on helping leaders get more done with fewer headaches by outlining clear, creative strategies and solutions that build momentum and buy-in at all organizational levels. In addition to consulting, I write regularly for Inc.com, GovExec.com, and Bloomberg Government on leadership and how to increase your positive impact at work. She is the author of a best-selling book on organizational behavior entitled, Flock, Getting Leaders to Follow and Own It: Drive Your Career to a Place of Happiness and Success. I live with my husband and three children in Falls Church, Virginia.

Avoiding Data Collection Boondoggles

Passing Time by Allison Long Hardy

Passing Time by Allison Long Hardy

Blinded by our love of metrics, we pursue data. Yet, as we recently explored, the cost of that pursuit is often high. Staff time is assigned, support contracts are signed, information technology systems are built and secure storage facilities are negotiated.

We are also blinded by an assumption: that data is critical to anticipating the future and investing public funds responsibly. We’re intrigued, entranced, obsessed. We worry about the gaps and suspected errors so we dismiss any extrapolated insights as flawed. And, then we come right back because—like all codependent relationships—we believe we need it (data) to exist.

In spite of this conundrum, we get sucked into the data game like we’re at a carnival. We go back time after time trying to get enough tickets to buy the big prize—but suspecting somewhere in the back of our minds that the stuffed animal is worth a fraction of what we spent to “win” it. And despite working in a time of notable budget shortfalls, the cost to collect data is rarely being scrutinized or the efforts to collect data cut back once they’re started.

Our data appetite is insatiable. Why? Because we’re desperate for answers. When we’re not exactly sure what the questions are, we believe data is the first step on the right path to getting there. In fact, you’d be hard-pressed to find a federal program manager (or their consultant behind the scenes egging them on) who didn’t think having more data was a good thing. Perspectives like that of Kevin Cincotta in Government Executive make so much sense on the surface that data collection efforts often go unchallenged.

Our data appetite convinces us that we’ll use it if we have it. In reality, nothing seems further from the truth. Forrester reported last year that a meager 12 percent of the data collected is ever analyzed. Yikes! This number might be startling at first glance, but it won’t really surprise many federal employees who live this reality. Within our federal programs, there is a limited awareness of the data available, concerns about data quality, a lack of analytical skill available to analyze it, and a just plain old lack of time. The time factor, to me, is the single biggest downfall with collected data. It’s expensive to collect and sadly, much of it goes unused. Plain and simple, it’s a waste.

So, we’re stuck, right? Maybe not. Responsible and assertive leaders must chart another path—up and around the hurdle created by the need for data. This path consists of three steps:

1. Build awareness

To start, agency leaders need a clear understanding of the data collection efforts underway within their organizations. This understanding doesn’t require an exhaustive inventory, but it does require a broad understanding of the top five or so programs requiring data collection and maintenance. Understanding what directives staffs are working under and what the response looks like—the number of staff assigned, the rough value of support contracts, and related systems. This exercise is necessarily conducted at the leadership level—it’s not a program by program problem.

Next, look at the internal and external reports being produced. Stretch beyond what information is being conveyed on the surface and dig deep to determine what (if any) insights are being gained. Which data elements do staffs depend on regularly to make decisions about the direction of the organization?

Based on the understanding gained by agency leaders in step 1, plot the big data collection efforts on a simple 2×2 matrix. Label one axis “federal mandate” (yes, no, or sort of) and the other “high mission utility” (yes, no, or sort of). Obviously, two combined “yeses” earns a green light, while two combined “noes” means an effort is phased out. The “sort of” rating equates to the work of program managers and leadership to either move certain data collection efforts solidly into one box or the other.

2. Vigorously push back:

For any data collection efforts falling into the “yes it’s a federal mandate, but has no mission utility” quadrant, agency leadership should vigorously push back, request an exception, or work to change the requirement so that it better fits the agency’s purpose. Agencies (and the government as a whole) simply don’t have the discretionary budget it takes to apply blanket rules to a deeply nuanced environment.

3. Reenergize analytics:

The biggest thing leaders can do to get more value out of the time invested in data collection is to ask for the reports. Ask program managers to share with you what they think is important. Be careful not to ask too loudly about what else would be nice to know—remember that anything you ask for has a trickle-down effect that costs more money.

Responsible leaders know the costs of data collection and continuously weigh the benefits. They also ask the right questions in the pursuit of data: Are we getting the insights we need? Are we getting the return on insights for the investment in collection? Would the public agree?

This article originally appeared in Bloomberg Government.

Robin Camarote

I'm the co-founder of Federal MicroConsulting and strategic planning consultant based on Falls Church, VA. I am intent on helping leaders get more done with fewer headaches by outlining clear, creative strategies and solutions that build momentum and buy-in at all organizational levels. In addition to consulting, I write regularly for Inc.com, GovExec.com, and Bloomberg Government on leadership and how to increase your positive impact at work. She is the author of a best-selling book on organizational behavior entitled, Flock, Getting Leaders to Follow and Own It: Drive Your Career to a Place of Happiness and Success. I live with my husband and three children in Falls Church, Virginia.