Are you tracking what matters most?

Performance metrics matter. We know this and yet we continue to track things that have little to do with our goals. The better alternative is to make sure that you're measuring progress against the things that matter most to you and your future. Here are four steps to think through that process.

I'm trying to move away from outputs-- such as number of contracts-- and focus more on outcomes. Did I create a connection with a client? Did a client come back after the project to ask more questions or start another task? Would we both want to work together again?

What do you track on a regular basis?

Only 12 Percent of Data Collected is Ever Used

 Emerald Summer by Lauren Adams

Emerald Summer by Lauren Adams

Wow. 

Govloop shared this stat in a post last and it got my attention.  If someone had asked me for the percent of data analyzed vs. what's collected, I would have guessed less than 100 percent but wouldn't have gone that low.  

The other thing that struck me was that-- for as much time as my client and team spends thinking about data-- we'd never examined their issues with field-level buy-in, demonstrating progress, or general forecasting from that perspective. So, I think it's a great, highly quote-able stat.

WHY I LOVE IT

It has shock value. Also, it came from a Forrester Research survey so it has credibility.  I did a little digging and found this reference on their site for more information.  It's unclear if this report on hadoop contains the survey data but if you're interested in the exact citation, that would be a place to start.

Here's an excerpt... "In a recent Forrester survey, technology execs and decision-makers ranked data-related projects at the top of their list for importance and investment. Why? Companies seek deeper insights from the massive amount of data at their disposal but estimate that they are analyzing only 12% of the data that they already have, leaving 88% of it on the cutting-room floor."

HOW YOU MIGHT USE IT

If you work with data-- and, really, who doesn't-- you can use this reference in two ways.  

  1. From an analytics perspective: Use this single little stat to launch a review of the body of data you/your client collects. Consider the data fields you collect as an entity and asset unto itself. Pull the list of fields into a spreadsheet, then make tabs for all of your common report queries with the lists of fields they're pulling.  Calculate the stat for your organization.  From the stuff that isn't utilized, figure out quickly if you can stop collecting it or find a way in the next 3 months to use it.  I suspect that a lot of these fields aren't used because some people are populating them and others aren't.  So there is no confidence that the data is sufficiently complete to be relied upon for any analysis.  To overcome this, just keep what you can use in the near-term and let folks off the hook for collecting everything else.
  2. From a communications perspective: Take this stat and the one you calculated for your organization and share this in your next leadership meeting. Outline your plan for increasing the use select fields and plan to sunset everything else. Both scenarios represent a win in data use and time savings for data elimination.

Robin Camarote

I'm the co-founder of Federal MicroConsulting and strategic planning consultant based on Falls Church, VA. I am intent on helping leaders get more done with fewer headaches by outlining clear, creative strategies and solutions that build momentum and buy-in at all organizational levels. In addition to consulting, I write regularly for Inc.com, GovExec.com, and Bloomberg Government on leadership and how to increase your positive impact at work. She is the author of a best-selling book on organizational behavior entitled, Flock, Getting Leaders to Follow and Own It: Drive Your Career to a Place of Happiness and Success. I live with my husband and three children in Falls Church, Virginia.

Data Visualization Inspiration

In this post, I shared one of my favorite examples of small multiples. Save the Children's report on the state of moms around the world focused in a couple of key numbers and repeated the pattern for each country included. 

Here's a totally different example from Simon Rogers Guardian and another very effective use of circles.

data visualization example robin camarote

 

 

In contrast to the small multiples, this graph highlights a subset of countries, organizes them by continent, then focuses on a single metric-- emission levels. 

WHY I LOVE IT

These circles are so effective because they enable the reader to quickly understand the difference in magnitude on a single data point. I like that it's grouped by continent because it helps orient you to where the lesser known countries are located. I also love the colors but it might leave some people craving Skittles rather than evoking concern for the environment.  Lastly, it's further supported by a couple of line graphs and tables at the bottom for some addition detail. Big stuff on the top, little on the bottom. 

HOW YOU MIGHT USE IT

If you like the circles*, consider using them to compare a single important data element across a number of units within your organization.  Program budgets might be one example.  Or people served by a specific program benefiting the public across a number of locations might be another. The key is to pick one important outcome and track it against multiple contributors so that you can see the key players.

To be sure, this took lots and lots of time to assemble. I don't know specifically but I'm guessing it was more than a single afternoon. If you're thinking of replicating a series of charts like these, consider doing it as a poster that will be printed and displayed, as well as, posted electronically. This format isn't appropriate for a PowerPoint presentation.  It should be treated as a stand-alone communications product. Actually, it's a good reason to forget the presentation all together and just do this.

*I had a client years ago who didn't like circles. I'm not kidding. They were banned from all deliverables after we brought him a draft that he glanced at then retitled "the birth control chart" during a big staff meeting. Awesome. I'd never met someone with such an aversion to a basic shape. Primary colors?  Sure.  A lot of people hate yellow.  But a circle?  It makes me sad just thinking about it. I hope this never happens to you.

Robin Camarote

I'm the co-founder of Federal MicroConsulting and strategic planning consultant based on Falls Church, VA. I am intent on helping leaders get more done with fewer headaches by outlining clear, creative strategies and solutions that build momentum and buy-in at all organizational levels. In addition to consulting, I write regularly for Inc.com, GovExec.com, and Bloomberg Government on leadership and how to increase your positive impact at work. She is the author of a best-selling book on organizational behavior entitled, Flock, Getting Leaders to Follow and Own It: Drive Your Career to a Place of Happiness and Success. I live with my husband and three children in Falls Church, Virginia.

When the Pursuit of Data Just Isn’t Worth It

 Linear Composition 3, by Andrew Mechum

Linear Composition 3, by Andrew Mechum

Data is everywhere and, for many, is everything. It’s put on a pedestal where it is both loved and admired. It’s protected and cared for by smart, devoted people. And, it’s both the question and the answer when we cross paths with the OMB or Congress.

I started to write that federal programs “these days” have become synonymous with massive data collection, analysis and reporting exercises, but we all know that’s not 100 percent true. For as long as we have been a nation, we (the public) have demanded from our government (and our contractors) a thorough accounting for the money spent and the accomplishments achieved. In today’s government, that means data and that’s all good.

The issue? A little taste for good data leads to cravings for more. Without realizing it, we’ve become zombies with insatiable appetites for fleshy spreadsheets. That’s not so good.

Why? Data is expensive—in fact, far more expensive than we like to acknowledge. In government and in contracting, data is the fancy, bubbly, bottled water that we treat like tap.

We rarely talk about the cost of data because we believe data makes us smarter and better organizations. Federal executives want insights and answers—so they ask for more analysis. Program managers with their staffs and consultants want to be responsive and to show their program’s value—so they collect data to power such analysis and start crunching. They might wince a little in the process of getting there, but ultimately they move heaven and earth to produce polished, data-rich reports.

Yet for the dozens of well-purposed, well-intentioned federal programs, such as the Federal Real Property Profile (FRPP), the Employer Information Report, the Energy Review, and federal information technology (IT) investments, data requests like these are not making them smarter or better. They are merely creating a data collection burden on both federal employees and their contractors.

Each of these programs (and the many others like them) has a purpose. And all the data they produce has potential value. But how much did it cost to obtain it? Where is that data now? And who is using it?

The FRPP exemplifies a good idea gone wrong. If you’re not familiar with this program, the FRPP is a skim of a federal agency’s real property data and metrics. Agency staff collect the data and send it to the GSA. Using that submitted data, the GSA then produces an annual summary report of the federal government’s footprint.

And then not much else happens.

Now many important things are happening in real property at each of the agencies that reports data—but the FRPP data at the GSA is not reflective of that activity. In fact, the rules around FRPP data make it too hard to easily snag data from other systems and too superficial to do much interesting, useful analysis with it on its own. Yet agencies still bear the cost of gathering the required data, checking it, and submitting it for no return on their data collection investment—except for a check in the compliance box.

In their book, the “Agile Culture,” Pollyanna Pixton, Paul Gibson, and Niel Nickolaisen advise, “Always ensure that the cost of collecting the metric is significantly less than the value that it can deliver. And we do mean significantly less.” Recognize that the cost of collecting data is not zero and can sometimes be very high, especially if it involves continuous action by the delivery team members. This cost is often ignored and can have a huge negative impact on team productivity.

Even good programs can grow increasingly expensive because we need—and want—to understand broad, complex problems. There is no doubt that our pursuit of data and answers has yielded some insights, avoided some crises and enabled some right choices being made the first time. Yet is it right, or even sustainable, to pursue data without periodically asking ourselves how much it is going to cost to obtain the data?

How do we think the data we seek is going to answer an unanswered question? Is the tradeoff between the expense to collect the data and insight gained worth it? If the answer to this quick “gut check” is yes, then by all means, pursue that data with purpose and seriousness. However, if the answer is less than a resounding “yes” or we’re struggling year after year to ensure collection compliance, maybe it’s time to admit that the data isn’t needed or as valuable as once thought. Seeking data at all costs isn’t a wise (or sustainable from a political or budgetary perspective) approach.

The alternative is to treat data like other investments and to measure its return on investment. Keep collection and refinement efforts in check by continuously weighing the costs and benefits.

Robin Camarote

I'm the co-founder of Federal MicroConsulting and strategic planning consultant based on Falls Church, VA. I am intent on helping leaders get more done with fewer headaches by outlining clear, creative strategies and solutions that build momentum and buy-in at all organizational levels. In addition to consulting, I write regularly for Inc.com, GovExec.com, and Bloomberg Government on leadership and how to increase your positive impact at work. She is the author of a best-selling book on organizational behavior entitled, Flock, Getting Leaders to Follow and Own It: Drive Your Career to a Place of Happiness and Success. I live with my husband and three children in Falls Church, Virginia.

Avoiding Data Collection Boondoggles

 Passing Time by Allison Long Hardy

Passing Time by Allison Long Hardy

Blinded by our love of metrics, we pursue data. Yet, as we recently explored, the cost of that pursuit is often high. Staff time is assigned, support contracts are signed, information technology systems are built and secure storage facilities are negotiated.

We are also blinded by an assumption: that data is critical to anticipating the future and investing public funds responsibly. We’re intrigued, entranced, obsessed. We worry about the gaps and suspected errors so we dismiss any extrapolated insights as flawed. And, then we come right back because—like all codependent relationships—we believe we need it (data) to exist.

In spite of this conundrum, we get sucked into the data game like we’re at a carnival. We go back time after time trying to get enough tickets to buy the big prize—but suspecting somewhere in the back of our minds that the stuffed animal is worth a fraction of what we spent to “win” it. And despite working in a time of notable budget shortfalls, the cost to collect data is rarely being scrutinized or the efforts to collect data cut back once they’re started.

Our data appetite is insatiable. Why? Because we’re desperate for answers. When we’re not exactly sure what the questions are, we believe data is the first step on the right path to getting there. In fact, you’d be hard-pressed to find a federal program manager (or their consultant behind the scenes egging them on) who didn’t think having more data was a good thing. Perspectives like that of Kevin Cincotta in Government Executive make so much sense on the surface that data collection efforts often go unchallenged.

Our data appetite convinces us that we’ll use it if we have it. In reality, nothing seems further from the truth. Forrester reported last year that a meager 12 percent of the data collected is ever analyzed. Yikes! This number might be startling at first glance, but it won’t really surprise many federal employees who live this reality. Within our federal programs, there is a limited awareness of the data available, concerns about data quality, a lack of analytical skill available to analyze it, and a just plain old lack of time. The time factor, to me, is the single biggest downfall with collected data. It’s expensive to collect and sadly, much of it goes unused. Plain and simple, it’s a waste.

So, we’re stuck, right? Maybe not. Responsible and assertive leaders must chart another path—up and around the hurdle created by the need for data. This path consists of three steps:

1. Build awareness

To start, agency leaders need a clear understanding of the data collection efforts underway within their organizations. This understanding doesn’t require an exhaustive inventory, but it does require a broad understanding of the top five or so programs requiring data collection and maintenance. Understanding what directives staffs are working under and what the response looks like—the number of staff assigned, the rough value of support contracts, and related systems. This exercise is necessarily conducted at the leadership level—it’s not a program by program problem.

Next, look at the internal and external reports being produced. Stretch beyond what information is being conveyed on the surface and dig deep to determine what (if any) insights are being gained. Which data elements do staffs depend on regularly to make decisions about the direction of the organization?

Based on the understanding gained by agency leaders in step 1, plot the big data collection efforts on a simple 2×2 matrix. Label one axis “federal mandate” (yes, no, or sort of) and the other “high mission utility” (yes, no, or sort of). Obviously, two combined “yeses” earns a green light, while two combined “noes” means an effort is phased out. The “sort of” rating equates to the work of program managers and leadership to either move certain data collection efforts solidly into one box or the other.

2. Vigorously push back:

For any data collection efforts falling into the “yes it’s a federal mandate, but has no mission utility” quadrant, agency leadership should vigorously push back, request an exception, or work to change the requirement so that it better fits the agency’s purpose. Agencies (and the government as a whole) simply don’t have the discretionary budget it takes to apply blanket rules to a deeply nuanced environment.

3. Reenergize analytics:

The biggest thing leaders can do to get more value out of the time invested in data collection is to ask for the reports. Ask program managers to share with you what they think is important. Be careful not to ask too loudly about what else would be nice to know—remember that anything you ask for has a trickle-down effect that costs more money.

Responsible leaders know the costs of data collection and continuously weigh the benefits. They also ask the right questions in the pursuit of data: Are we getting the insights we need? Are we getting the return on insights for the investment in collection? Would the public agree?

This article originally appeared in Bloomberg Government.

Robin Camarote

I'm the co-founder of Federal MicroConsulting and strategic planning consultant based on Falls Church, VA. I am intent on helping leaders get more done with fewer headaches by outlining clear, creative strategies and solutions that build momentum and buy-in at all organizational levels. In addition to consulting, I write regularly for Inc.com, GovExec.com, and Bloomberg Government on leadership and how to increase your positive impact at work. She is the author of a best-selling book on organizational behavior entitled, Flock, Getting Leaders to Follow and Own It: Drive Your Career to a Place of Happiness and Success. I live with my husband and three children in Falls Church, Virginia.