You will be redirected to the page you want to view in  seconds.

Big Data: How to fulfill the promise of limitless intelligence

Sep. 19, 2013 - 03:46PM   |  
By JEFF SORENSON   |   Comments
Jeff Sorenson is president and partner, A.T. Kearney Public Sector & Defense Services, and former Army CIO/G6.
Jeff Sorenson is president and partner, A.T. Kearney Public Sector & Defense Services, and former Army CIO/G6. (File)
  • Filed Under

Big data is creating new business models, spurring increased growth and cost-reduction capabilities, and driving competitive advantage for companies throughout the commercial sector. In much the same way, it has the potential to transform the way the Department of Defense accomplishes its mission.

To benefit from big data, though, the DoD must learn from the private sector how to efficiently and cost-effectively leverage the information and how to overcome the challenges that prohibit it from taking advantage of big data’s promise of limitless intelligence.

In the past, structural and systemic limitations — incomplete data sets, lack of analytical capabilities — restricted the ability of business enterprises and the DoD to push the insights they could garner from the data they collected. With today’s ever-increasing volume, variety and velocity of data, and the commercial sector’s technological and analytic capability to smartly mine this big data, businesses are fundamentally changing the way they make decisions, ways that only recently would have seemed like science fiction speculations. For example:

■ Some financial institutions use algorithmic trading to analyze minute-by-minute market data and to gain new opportunities to create value.

■ A retailer uses in-store video and Wi-Fi data to follow customers through the aisles and then engages them with items that complement their purchases.

■ Another retailer analyzes sell-through rates, out-of-stocks, and price promotions at the SKU level and then creates probability scenarios for selling a selected item at a specified place or time.

■ One manufacturer now builds “connected” products that send massive amounts of data over the Internet to product engineers who use it to make the company’s products more efficient, saving customers billions of dollars each year.

The DoD is wrestling with this same issue of how best to mine precise information from all the tactical and battlefield data it gathers and stores.

However, some estimates suggest the DoD now analyzes less than 10 percent of the data available for a mission. Sometimes that percentage is sufficient — if the data available are large enough and worked hard enough to overcome data-processing inefficiencies.

But how much better could the DoD target and garner key information about our adversaries and their potential actions if it used all available data? Unfortunately, in many cases, the DoD has not kept up with the process methodologies employed by successful commercial companies for drawing the precise information needed from their data stockpile.

Overcoming the challenges

Businesses that have succeeded in using big data to improve their competitive advantage learned how to overcome four challenges that the DoD and other companies must overcome:

Access Complexity: This complexity is two fold: (1) a proliferation of data sources that provide a variety of structured and unstructured data, and (2) multiple tools, technologies or protocols used by different parts of the organization to manage the data and attempt to generate insights from it.

Success in achieving limitless intelligence comes from learning how to think through this complexity in a smart way.

Stakeholder and Data Silos: Separate stakeholders within the organization often maintain their own data silos and refuse to share data across departments or value chain partners. This creates an absence of standardized data capture, reporting systems and processes.

The organization’s challenge is to organize this data — no matter where it’s generated — into a single, integrated database that can be accessed by all to analyze the best course of action.

Rapidly Evolving Ecosystem: In the big data space, the technology and analytical assets available to organizations are quickly and constantly changing, as both established players and sophisticated startups drive leading-edge thinking.

Within this dynamic landscape capital investment decisions are increasingly difficult. Are new capabilities worth the disruption caused to employees or staff comfortable with the current way of doing things? Should organizations attempt first-mover advantage or wait to see if the new capabilities will really lead to the best insights?

Unclear Value Proposition: At the conceptual level, everybody agrees that big data can help organizations make better and faster decisions. In reality, however, many also are unsure what big data’s value proposition is really worth. Although big data might enable the DoD to make a slightly better decision a second or two faster, is it worth trading a hard physical asset (say, a tank) for this multimillion dollar capability? What are some criteria to help make this decision?

Successful businesses know how to capture, identify and quantify this value proposition for their stakeholders who must buy in to big data, and their insights might prove helpful as the DoD starts thinking about this task.

Adapting lessons learned

As this post points out, successful businesses continue to learn creative and innovative lessons that help them realize the unlimited intelligence promised by big data. By adapting what they’ve learned, the DoD can efficiently and effectively use big data to drive better decision-making.

The next three posts will explore practices and lessons the DoD can leverage from the commercial sector:

■ How should the organization go about securing, storing and syndicating data for key decision making?

■ An explosion of data channels has led to a saturation of structured and unstructured data. How have organizations converted this data into actionable, predictive insights?

■ Often the data that an organization needs to inform its decisions is common and shareable. What are the governance models that can be used to leverage data, technology and analytic assets across the organization?

Jeff Sorenson is president and partner, A.T. Kearney Public Sector & Defense Services, and former Army CIO/G6.


Start your day with a roundup of top defense news.

More Headlines



Login to This Week's Digital Edition

Subscribe for Print or Digital delivery today!

Exclusive Events Coverage

In-depth news and multimedia coverage of industry trade shows and conferences.



Defensenews TV

  • Sign-up to receive weekly email updates about Vago's guests and the topics they will discuss.