Abstract
An essential component to intelligence analysis is inferring an explanation for uncertain, contradictory, and incomplete data. In order to arrive at the best explanation, effective analysts in any discipline conduct an iterative, convergent broadening and narrowing hypothesis assessment using their own tradecraft. Based on this observation, we developed an adaptive framework to support intelligence analysis while being tradecraft agnostic. The Reasoning About Multiple Paths and Alternatives to Generate Effective Forecasts (RAMPAGE) process framework provides a structure to organize and order analysis methods to maximize the number and quality of hypotheses generated, helping to improve final forecasts. The framework consists of five stages of analysis: (1) Information Gathering and Evaluation; (2) Multi-Path Generation; and (3) Problem Visualization; (4) Multi-Path Reasoning; and (5) Forecast Generation. As part of IARPA’s FOCUS program, we demonstrated the flexibility of this framework by developing five versions of the process to answer five different sets of counter-factual forecasting challenges. While the FOCUS program concentrated on counter-factual forecasting, this framework was designed to support hypothesis generation and assessment, which is a critical component of analysis across the intelligence domain.
This is a preview of subscription content, log in via an institution.
Buying options
Tax calculation will be finalised at checkout
Purchases are for personal use only
Learn about institutional subscriptionsReferences
Patterson, E.S., Roth, E.M., Woods, D.D.: Predicting vulnerabilities in computer-supported inferential analysis under data overload. Cogn. Technol. Work 3, 224–237 (2001)
Elm, W., Potter, S., Tittle, J., Woods, D., Grossman, J., Patterson, E.: Finding decision support requirements for effective intelligence analysis tools, pp. 297–301. SAGE Publications, Los Angeles (2005)
Roth, E.M., et al.: Framing and contextualizing information requests: problem formulation as part of the intelligence analysis process. J. Cogn. Eng. Decis. Making 4, 210–239 (2010)
Elstein, A.S., Schwarz, A.: Clinical problem solving and diagnostic decision making: selective review of the cognitive literature. BMJ 324, 729–732 (2002)
Trent, S., Voshell, M., Patterson, E. Team cognition in intelligence analysis, pp. 308–312. SAGE Publications, Los Angeles (2007)
Asare, S.K., Wright, A.M.: A note on the interdependence between hypothesis generation and information search in conducting analytical procedures. Contemp. Account. Res. 20, 235–251 (2003)
Bailey, C.D., Daily, C.M., Phillips, T.J., Jr.: Auditors’ levels of dispositional need for closure and effects on hypothesis generation and confidence. Behav. Res. Account. 23, 27–50 (2011)
Sprenger, A., Dougherty, M.R.: Generating and evaluating options for decision making: the impact of sequentially presented evidence. J. Exp. Psychol. Learn. Mem. Cogn. 38, 550 (2012)
Thomas, R., Dougherty, M.R., Buttaccio, D.R.: Memory constraints on hypothesis generation and decision making. Curr. Dir. Psychol. Sci. 23, 264–270 (2014)
Sprenger, A.M., et al.: Implications of cognitive load for hypothesis generation and probability judgment. Front. Psychol. 2, 129 (2011)
Dougherty, M.R., Gettys, C.F., Thomas, R.P.: The role of mental simulation in judgments of likelihood. Organ. Behav. Hum. Decis. Process. 70, 135–148 (1997)
Lange, N.D., Thomas, R.P., Davelaar, E.J.: Temporal dynamics of hypothesis generation: the influences of data serial order, data consistency, and elicitation timing. Front. Psychol. 3, 215 (2012)
Lorenz, J., Rauhut, H., Schweitzer, F., Helbing, D.: How social influence can undermine the wisdom of crowd effect. Proc. Natl. Acad. Sci. 108, 9020–9025 (2011)
Klayman, J.: Varieties of confirmation bias. In: Psychology of Learning and Motivation, pp. 385–418. Elsevier (1995)
Lehner, P.E., Adelman, L., Cheikes, B.A., Brown, M.J.: Confirmation bias in complex analyses. IEEE Trans. Syst. Man Cybern. Part A Syst. Hum. 38, 584–592 (2008)
Dougherty, M.R., Hunter, J.E.: Hypothesis generation, probability judgment, and individual differences in working memory capacity. Acta Physiol. (Oxf) 113, 263–282 (2003)
Dougherty, M., Thomas, R., Lange, N.: Toward an integrative theory of hypothesis generation, probability judgment, and hypothesis testing. In: Psychology of Learning and Motivation, pp. 299–342. Elsevier (2010)
Klein, G., Phillips, J.K., Rall, E.L., Peluso, D.A.: A data-frame theory of sense making, pp. 113–155. Lawrence Erlbaum, New York (2007)
Peabody, M., Veinott, E.S.: Focus shift: differences in reasons generated using Premortem and Worst Case Scenario plan evaluation methods. ndm 259 (2017)
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2021 Springer Nature Switzerland AG
About this paper
Cite this paper
McDermott, A.F. et al. (2021). Developing an Adaptive Framework to Support Intelligence Analysis. In: Sottilare, R.A., Schwarz, J. (eds) Adaptive Instructional Systems. Design and Evaluation. HCII 2021. Lecture Notes in Computer Science(), vol 12792. Springer, Cham. https://doi.org/10.1007/978-3-030-77857-6_39
Download citation
DOI: https://doi.org/10.1007/978-3-030-77857-6_39
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-030-77856-9
Online ISBN: 978-3-030-77857-6
eBook Packages: Computer ScienceComputer Science (R0)