The operators at a natural gas control center in downtown San Francisco weren’t sure whether to believe the spike in pipeline pressures they were seeing along the length of the San Francisco Peninsula. Technicians working at a natural gas hub in Milpitas, 40 miles away, had warned that electrical work might briefly affect readings from their supervisory control and data acquisition equipment. Sure enough, the work had temporarily cut off all SCADA data on the pressures and valve positions at the Milpitas Terminal. The operators had no way of knowing what investigators would later discover: A voltage drop had caused an erroneous low-pressure reading, and in response, some valves at Milpitas had automatically opened.
With high-pressure alarms now sounding in San Francisco, one operator decided the readings looked real while another rushed to call for help: “We’ve over-pressured the whole peninsula,” the operator said. Before operators could do anything, a 54-year old pipe with faulty welds ruptured under a San Bruno neighborhood. The resulting explosion killed eight people and destroyed 38 homes.
The September 2010 San Bruno disaster was not caused by cyber criminals or terrorists, but U.S. security experts say, it illustrates the kinds of calamity that hackers could cause by injecting malware into the complex electronics and software that run the country’s physical infrastructure. 2012 is shaping up as a possible breakthrough year for those who have been trumpeting this kind of threat.
Army Gen. Keith Alexander, director of the National Security Agency and chief of U.S. Cyber Command, has been speaking with fresh bluntness about the SCADA risk, and Congress is weighing legislation that would either require private companies and utilities to make fixes, or try to inspire those fixes through tax credits.
In this new environment, cybersecurity companies are jockeying to define the strategies and technologies that would do the best job of augmenting the safeguards on SCADA devices. The most common type of industrial control system, SCADA systems are used throughout the world to remotely control activities in nuclear power plants, water treatment facilities, hydroelectric dams, natural gas and oil pipelines.
“There is nothing hypothetical about the problem,” said Joseph Weiss, managing partner of Applied Control Solutions and author of the book “Protecting Industrial Control Systems from Electronic Threats.” The failures of industrial control systems have triggered “well over 200 incidents in the last decade,” he said, citing three events that caused U.S. nuclear power plants to halt operations.
The Stuxnet computer worm caused the most well-known SCADA breach when it caused Iranian nuclear fuel centrifuges to spin out of control and break in 2010. In September, computer security experts in Budapest, Hungary, identified a derivative of Stuxnet, called Duqu, that is designed steal data. “The attackers are looking for information such as design documents that could help them mount a future attack” on various industries, including industrial control system facilities, according to a white paper released by Symantec Corp. in November.
When nCircle Network Security Inc. asked people attending major computer security conferences whether they expected “a significant SCADA breach in 2012,” 75 percent of the people surveyed at the Black Hat Europe conference in Amsterdam in March said yes and 48 percent of the people at the RSA conference in San Francisco agreed.
U.S. officials worry that American sites could be next. Alexander told lawmakers that adversaries who once stole data or disrupted Internet activity are learning to hack into computers to cause physical damage. “Attacks that can destroy equipment are on the horizon, and we have to be prepared for them,” he said March 20 at a hearing of the House Armed Services subcommittee on emerging threats and capabilities. At the same hearing, Madelyn Creedon, assistant secretary of defense for global strategic affairs, said the Defense Department relies on the U.S. electric grid, as well as key transportation and communications services.
“Unless we, as a nation, do more to protect critical infrastructure assets and intellectual property, it is likely only a matter of time before we suffer a crippling blow that will greatly diminish DoD’s ability to conduct our missions,” she said.
The U.S. government wants to find solutions, but implementing them won’t be easy. One reason is the variety of operating schemes for SCADA systems and site-specific software called Distributed Control Systems. SCADA systems generally operate equipment dispersed over wide geographic areas. Distributed Control System software must be installed at individual sites to automate processes, including those at power generation plants. SCADA software sometimes links sites programmed with distributed control software. This software is so varied and complex that security cannot be achieved through traditional software patches or firewalls. Instead, the best approach will be to assess individual control systems and draft plans to shore up vulnerabilities, security experts said.
The government might also end up playing a part. Congressional committees are weighing legislative proposals to tighten security for key facilities with a carrot-and-stick approach, and Pentagon planners say they are willing to share intelligence on threats to industrial control systems.
For decades, industrial control systems were designed as stand-alone, analog devices used to monitor and control everything from oil pipelines to the chemical composition of pesticides. Manufacturers eventually decided to ease manpower demands and monitoring activity by adding computers to link the industrial control systems to the Internet or to local or wide area networks. When they did that, according to security experts, they did not include safeguards to make those systems difficult to penetrate.
Even today, the people running power plants or wastewater treatment facilities are not focused on security, said Elizabeth Ireland, nCircle’s vice president for marketing. They are focused on keeping the plant running as efficiently as possible, 24 hours a day, 365 days of the year.
Adding security after the fact can be challenging. Security professionals said the best way to improve the safety of industrial control systems is to conduct a thorough assessment of the role each one plays in a facility; determine whether failure of that system would jeopardize lives and property; review safeguards already in place to mitigate risk; and develop a plan to close any gaps.
Still, executives from network security companies say many efforts to safeguard industrial control systems will share common elements. For example, McAfee is working with SCADA system manufacturers to augment security through a process called application whitelisting, in which security experts generate a list of the operations a SCADA system is allowed to perform.
“All the instructions in the world can come to my machine, but I know which ones are approved,” said Phyllis Schneck, McAfee’s public sector chief technology officer. Sending malware to a SCADA system featuring applications whitelisting is “like sending a Windows virus to a Mac. It might get in, but it can’t cause any harm.”
To bolster security, facility operators also need to make sure that every element of an industrial control system produces activity logs, said Weiss. Without those logs, it is extremely difficult to spot anomalous activity or trace the chain of events leading up to any problems.
Another way to address security is through testing. Wurldtech of Vancouver, British Columbia, has developed a way to pinpoint the vulnerabilities of SCADA systems with an electronic box it calls the Achilles test platform. Achilles is programmed with simulated malware. When plugged into a SCADA system, Achilles floods it with traffic, including valid data and malware, while monitoring whether the system can continue to perform its job. “When we do find vulnerability within the controller, we work with customers to help them determine the cause, affect and mitigation strategies,” said Greg Speakman, Wurldtech’s marketing vice president. When SCADA systems pass Wurldtech’s test, the company awards them the Achilles Communications Certification, a document vendors can use to prove that their systems can stand up to external threats.
Although there have been many efforts by the government and industry to share threat and malware information, those initiatives have been largely unsuccessful, said Nate Kube, Wurldtech’s founder and chief technology officer. When companies discover vulnerabilities, they do not want to tell the world because it makes their systems less secure. Similarly, government officials often are wary of sharing classified information on new threats.
Even so, Alexander told lawmakers the government needs to help secure key facilities by sharing classified information on malware, viruses and other threats. The Pentagon launched a cybersecurity pilot program in 2011 to share information on classified signatures and methods with defense contractors and Internet service providers. While some industry participants panned the program early on, it has since been improved to provide companies with more timely access to classified threat information, Alexander said.
Other federal agencies also are strategizing to help safeguard industrial control systems. The Department of Energy requires power companies to take steps to safeguard SCADA systems through its designated agent, the North American Electric Reliability Corp. NERC’s critical infrastructure protection standards require companies to assess and address risks, including risks to SCADA systems. NERC can impose fines of $1 million per infraction per day on companies who fail to comply with the standards. Nevertheless, many companies are not complying. “Large utilities are spending enormous amounts of money coming up with legal arguments as to why their systems do not fall under this regulation rather than taking steps to make their systems secure,” a security professional said. For other industries, the government has established voluntary guidelines for securing industrial control systems. That’s been largely ineffective, Weiss said.
Congressional legislation seeks to impose new security standards on the private companies that own and operate 90 percent of the nation’s critical infrastructure. A Senate bill, the Cybersecurity Act of 2012, would direct the Department of Homeland Security to work with industry groups and federal agencies to identify those facilities that are essential to national security, U.S. economic security, public health and public safety. “The bill puts requirements on the operators of infrastructure, like the BPs of the world,” Kube said. “They push their vendors to supply solutions to help them meet those requirements.“
Other bills introduced in the House and Senate favor incentives instead of regulations to encourage companies to safeguard critical facilities. Tom Gann, McAfee’s vice president of government relations, favors that approach. By providing tax credits or reforming liability laws, the government can spur companies to improve security without imposing new regulations or fines, he said.
New regulations also may be difficult to pass this year because Congress is so polarized. Some Republicans in the House and Senate oppose cybersecurity legislation that seeks to impose any new regulations on industry.
“But this is not a partisan issue,” Weiss said. “The question is, do you want lights or not? Do you want water or not? Do you want another San Bruno or not?”