Risky business: what we have yet to learn about risk management
Section snippets
What's wrong with this picture?
To see that there is room for improvement, consider the flaw in the Pentium chip, reported in 1994. At the time the flaw was acknowledged, six million personal computers relied on the flawed chip. At $300 per chip, Intel’s risk impact was $1.8 billion, which includes not only the 3–4 million PCs already sold but also the remainder in stores and warehouses (Markoff, 1994). Intel’s risk assessment showed that “average” computer users would get a wrong answer (due to the chip’s flaw) every 27 000
What is risk management?
To understand how to improve our risk assessment expertise, we must first investigate how we are being told to evaluate risk today. We begin by asking how to determine what these risks are. For instance, guidance is provided in many places: books, articles, tutorials, and tools. Most advice asks us to distinguish risks from other project events by looking for three things (Rook, 1993 and Pfleeger, 1998):
- 1.
A loss associated with the event. The event must create a situation where something negative
Risk management activities
Software engineering textbooks and articles, such as Boehm (1991), Rook (1993), and Pfleeger (1998), lay out the steps of risk management, often using charts such as the one in Fig. 1. First, you assess the risks on your project, so that you understand what may occur during the course of development or maintenance. The assessment consists of three activities: identifying the risks, analyzing them, and assigning priorities to each of them. To identify them, you may use many different techniques.
Avoid false precision
Quantitative risk assessment is becoming more and more popular, both because of its inherent appeal to scientists and because it is often mandated by regulatory agencies. For instance, from 1978 to 1980, only eight chemicals were regulated on the basis of quantitative risk analysis in the US. But from 1981 to 1985, 53 chemicals were regulated that way. Similarly, there are more and more calls for quantitative assessments of software risk.
One of the first things to notice about how the rest of
Don’t be fooled by “questionable science”
Related to these issues of quantification and precision are issues of the science used to collect, analyze and present quantified risk information. The most problematic aspect of quantifying risk data is the possibility of misleading regulators and decision-makers into thinking that they can ignore or give less credence to qualitative data. That is, a numerical description of risk is often given more credence than a qualitative one, even when quantitative descriptions are known to be suspect.
Separate facts from values when you can
A final problem with conventional risk assessment is that it can never be value-free; the way we view the world colors our interpretation of facts. “The conviction that risk assessment can never be a value-free exercise led an NRC [US Nuclear Regulatory Commission] committee to recommend in 1983 that the functions of risk assessment and risk management should not be institutionally separated in the regulatory process, even though agencies should seek as far as possible to prevent
Next steps
There is risk in not paying attention to what others have learned from managing risk in other disciplines. As scientists, we like to think that we can make objective, accurate assessments of our projects’ risks, and then deal with them in a fair and effective way. But in reality, there is much fuzziness and uncertainty associated with the risks themselves and with our understanding of how to address them. So what can we do to give our clients and ourselves more confidence in our risk
Acknowledgements
I am grateful to Linda Greer of the Natural Resources Defense Council for her assistance in examining environmental risk management. A summary of problems in environmental risk management can be found in Rachel’s Environment and Health Weekly, #420, Environmental Research Foundation, PO Box 5036, Annapolis, MD 21403.
References (23)
Software risk management: principles and practices
IEEE Software
(January 1991)Informed consent, clinical research, and the practice of medicine
Transactions of the American Clinical and Climatological Association
(1982)- Commission of the European Communities., 1991. Benchmark exercise on major hazard analysis, vol. 3,...
- Fisher, L., 1994. Pentium flaw creates confusion for PC buyers, New York Times, December 14, pp. D1,...
- et al.
Clinical research in general medical journals: a 30-year perspective
New England Journal of Medicine
(1979) - IBM, 1994. IBM halts shipments of Pentium-based personal computers based on company research, press release, December...
The Fifth Branch: Science Advisors as Policymakers
(1990)- Jasanoff, S., 1991. Acceptable evidence in a pluralistic society, In: Deborah, M., Rachelle, H. (Eds.), Acceptable...
Sources of Power: How People Make Decisions
(1998)- et al.
An investigation of the Therac-25 accidents
IEEE Computer
(July 1993)
Cited by (52)
Analyzing the role of cognitive and cultural biases in the internalization of information security policies: Recommendations for information security awareness programs
2015, Computers and SecurityCitation Excerpt :Slovic et al. (1984) classify several risks in terms of known or unknown degree and dread or non-dread degree and argue that risks which are perceived as dread and unknown are overestimated, whilst non-dread and known risks are underestimated. A set of criteria to categorize risks are presented in Table 3 (Pfleeger, 2000). The psychometric paradigm allows us to understand why certain risks, such as skiing accidents and swimming pool accidents, are underestimated.
Assessing the stability of suppliers using a multi-objective fuzzy voting data envelopment analysis model
2022, Environment, Development and SustainabilityA Systematic Literature Review on Risk Assessment and Mitigation Approaches in Requirement Engineering
2021, Research Anthology on Agile Software, Software Development, and TestingAn Early Multi-Criteria Risk Assessment Model: Requirement Engineering Perspective
2021, Research Anthology on Agile Software, Software Development, and TestingRisk Management in Global CRM IT Projects
2020, Business Perspectives and ResearchA systematic literature review on risk assessment and mitigation approaches in requirement engineering
2019, Crowdsourcing and Probabilistic Decision-Making in Software Engineering: Emerging Research and Opportunities