TAMPA, Fla. – U.S. Special Operations Command is struggling to develop and implement technology that will help get a handle on the large amount of information it must sift through to stay informed, make decisions and execute operations.

And SOCOM is not alone in the struggle. It's a problem that plagues U.S. Armed Forces as a whole.

But mastering "big data" is imperative for special operations activities that rely on real time intelligence and very early indicators to make crucial life or death maneuvers. Leaders from around the command called upon industry for help mastering the seemingly endless data pouring in to be used for intelligence at the Special Operations Forces Industry Conference Tuesday.

The message echoed that of USSOCOM Commander Gen. Raymond Thomas in recent testimony: "We're dealing, literally swimming, in the morass of information and intelligence, a mixed bag," he  told a House Armed Services Emerging Threats and Capabilities subcommittee hearing earlier this month.

"But how we sort through that in terms of business solutions – we're on the cusp of it," Thomas said. "And the good news is, we're starting to marry up the right people with our operators and our problem solvers to get at this wicked problem of information management and deep data, all the things that go with it that, arguably, corporations have already addressed."

When asked at SOFIC how the command might be developing requirements for managing big data, Maj. Gen. Sean Swindell, director of USSOCOM’s Force Structure, Requirements, Resources and Strategic Assessments (J-8), said, "I don’t know that we have all the answers to that. That is currently something we are working right now," he said.

The command is "trying to make sure we have a unified way ahead so we don’t all have separate competing initiatives," across the components, Swindell said, noting that a "data council" has been set up to help tackle the task of generating recommendations. And once recommendations are ready for prime time, Swindell said he would push them forward through the chain for possible approval and execution.

Critical to the effort is collaborating with industry. "One thing we are hoping is to work with industry, to tell you the truth, to look at big data," Swindell added.

"I don’t think we have a big data problem, compared to what industry has to deal with, but it’s a big data problem to us for sure," Col. Michelle Schmidt, the USSOCOM’s director of intelligence (J-2) added.

But since the commercial world deals with big data on a level exponentially larger than USSOCOM, learning from industry is vital, USSOCOM leadership said.

SOCOM retrieves significant amounts of unclassified data during typical reconnaissance missions and all of the data is sent to a command center for exfiltration. S&T efforts need to address how to pull out the relevant and crucial information from large swaths of data streaming in and pass it on to intelligence analysts for further exploitation, according to the presentation.

Several companies presenting during an innovation showcase at SOFIC were in the data analytics business.

Cloudera, an open source data hub, can take data and run large-scale analytics. The company already digests data from NASA’s Orion Spacecraft and also developed the analytics process for fleet health management for Sikorsky’s aircraft, to name a few.

And Boston-based DataRobot provides an automated machine learning platform with 130 million predictive models built on its cloud platform, according to Kevin Stofan, a data scientist with the company.

Using artificial intelligence, machine-learning and technology that can process, eliminate the white noise and disseminate the important and usable intel, will be critical in managing big data in the future, intelligence analysts throughout the Pentagon and with the services have made clear.

But there’s always a matter of money needed to tackle the challenge. "When I hear all the talk about big data," USSOCOM’s comptroller Mark Peterson said, "I have a big money problem."

Therefore, Peterson noted, it would be ideal to serve as a test bed for the U.S. Army or another service. An effort such as developing technology that would ease the daunting process of big data analytics might be too big to try out with a service with data sets as large as the Army's, for example.

"We can test a proof-of-concept and if the service likes it they can grab it and then I can say it’s service common," which would alleviate some of the funding strain on SOCOM, Peterson added.