The Road to Vulnerability
Fri, 15 Aug 2003 17:09:33 -0700
One lesson that can be drawn from incidents like the recent massive power outage is that decreasing margins in all our infrastructures place critical societal functions at greater and greater risk of significant disruptions from rare accidental and malicious acts. Redefining acceptable levels of risks and protections as the world changes is hard work, but need to be done.
Cost pressures and tight engineering under benign assumptions lead to thin margins. Optimized engineering leads to most events being of small consequence (we’ve engineered systems to tolerate them), but some rare events can cause massive disruption. It would be ‘bad engineering’ to overdesign a system to tolerate very rare events, if that tolerance costs more than the failures it would prevent (in expected value to customer terms). Fragility to extremely rare events can be seen as good business. It would be surprising if there weren’t rare disruptions (like massive power outages) in highly optimized infrastructures.
But the invisible hand of economics and good engineering leave systems designed and optimized under assumptions of relatively benign environments at great risk if new or unexpected threats arise.
Computer systems change very rapidly, and new threats arise with disturbing speed. The current hardware manufacture, software development, and people practices of our cyber infrastructure are obviously subject to the same economic motivations as described above. So they are already (and will become even more) fragile to rare or unexpected accidental or malicious events. That’s ‘good business’ paving the road to vulnerabilities.
Post 9/11, we can point out how previously almost unthinkable scenarios are more thinkable now, and thus engineered defenses against potential attacks are more strongly motivated. Govt procurement practices, corporate and individual liability, government mandates, and other mechanisms could have a profound impact on the reliability and cost of cyber infrastructure, but also on large-scale economic concerns, so it may be imprudent to act without defining the threats. To define and quantify cyber threats and their impact, particularly in combination with coordinated physical and psychological attacks and effects, requires deep (read: expensive) contemplative research, development, large experimentation, etc. Once new threats and defenses are defined, all the costs associated with deployment of those mechanisms can be at least partially quantified, and then well-reasoned decisions can be made about appropriate levels of protection against various risks. The pace of technology change and societal reliance on these systems amplify the uncertainty, urgency, and magnitude of risk here. It is almost unthinkable that western societies would not put very large resources against a problem of this grave potential.