"The question will be the particular role of the board, where it will sit institutionally and how its recommendations might be acted on," FitzGerald said. "Even the traditional defense boards have a mixed record in terms of efficacy and utility."
The second, and perhaps most exciting initiative for the broader tech community, was the announcement of a "bug bounty" program that involves the Pentagon selecting a group of hackers, giving them department targets, and asking them to do as much damage as possible — and then reporting back what they found to help the DoD patch those holes, in exchange for some kind of reward.
Participants will be need to be registered and vetted, although a senior defense official said that the process for how they will be tested is still being worked out. All participants must be US citizens.
That type of program has become a best practice in the tech world, with the website BugCrowd.com maintaining a list of more than 470 companies that have such programs, including giants like Google, Microsoft, PayPal and Yahoo.
Jonathan Cran, vice president of operations at Bugcrowd, wrote in an email that the Pentagon announcement is a "great step in the right direction to addressing the critical need for cyber security skills in the US," but raised concerns about the US citizenship requirement to participate.
"In general, researcher talent is more expensive in the US, so limiting the program to US-based, background-checked researchers may present challenges or simply require more incentives to participate," Cran wrote. "33% of Bugcrowd's researcher base is here in the US, and less than 10% of those submit to background checks."
Mark Ryland, chief solutions architect with Amazon Web Services in Seattle, said Carter's outreach to the tech community has not gone unnoticed.
"Just this week seems like a real breath of fresh air," Ryland said.
DJ Patil, US chief data scientist and a former top executive at Silicon Valley heavyweights like eBay and LinkdIn, said he believes Silicon Valley will "100 percent" respond to Carter's overtures, particularly the bug bounty program.
"The number one thing I found as a Silicon Valley person is, everybody wants to figure out how to help. They struggle to figure out how you actually do that," Patil said. "What they're realizing now is 'oh, through DIUX, through all these other mechanisms, this is no longer talk – this is real.' And that is brokering a very different dialogue and conversation."
He also notes that hackers may take part in the bounty program even if the financial reward is limited, whether out of a sense of duty or a sense of pride at being able to discover flaws others had yet to identify.
Adds Peter Singer of the New America Foundation, "It is a great example of Pentagon following the practices of the best firms, building incentives to make the hacker market work for you. [It's a] great illustration of how the payoff of the outreach to Silicon Valley is not to be measured in new widgets but new ideas."
The point about ideas versus physical technology is a key one, especially when discussing the core of Carter's Silicon Valley outreach program, the Defense Innovation Unit-Experimental (DIUX).
That was on display during Carter's visit, when he held a "Shark Tank" style event where five small companies pitched him directly on their products for the Pentagon.
"It's very cool. It's very un-governmental," Naum said. "You expect it to be big bureaucracy. This is nimble, it's agile, and they actually do care."
And second, it needs to be set up to outlast Carter himself.
For his part, Carter said he was "very confident" the process will outlive him, in part because of the people who have been hired, and in part because focusing on innovation simply makes sense for the department.
FitzGerald is less confident, given Carter's role as the driver of the innovation discussion.
"The big question that remains for these announcements and the secretary's innovation agenda writ large is how it will persist beyond his tenure and, relatedly, to what extent will he be able to drive change in core DoD organizations and processes," FitzGerald said. "Those efforts require the personal support of the secretary. With less than a year remaining in his tenure, how will the secretary lock in his gains and truly move the building?"
Aaron Mehta was deputy editor and senior Pentagon correspondent for Defense News, covering policy, strategy and acquisition at the highest levels of the Defense Department and its international partners.