Skip to main content

Advertisement

SpringerLink
Log in

Some futures for cognitive modeling and architectures: design patterns for including better interaction with the world, moderators, and improved model to data fits (and so can you)

  • Manuscript
  • Published:
Computational and Mathematical Organization Theory Aims and scope Submit manuscript

Abstract

We note some future areas for work with cognitive models and agents that as Colbert (I am America (and so can you!), 2007) notes, “so can you”. We present three approaches as something like design patterns, so they can be applied to other architectures and tasks. These areas are: (a) Interacting directly with the screen-as-world. It is now possible for models to interact with uninstrumented interfaces both on the machine that the model is running on as well as remote machines. Improved interaction can not only support a broader range of behavior but also make the interaction more accurately model human behavior on tasks that include interaction. Just one implication is that this will force models to have more knowledge about interaction, an area that has been little modeled but essential for all tasks. (b) Providing the cognitive architecture with more representation of the body. In our example, we provide a physiological substrate to implement behavioral moderators’ effects on cognition. Cognitive architectures can now be broader in the measurements they predict and correspond to. This approach provides a more complete and theoretically appropriate way to include new aspects of behavior including stressor effects and emotions in models. And (c) using machine learning techniques, particularly genetic algorithms (GAs), to fit models to data. Because of the model complexity, this is equivalent to performing a multi-variable non-linear stochastic multiple-output regression. Doing this by hand is completely inadequate. While there is a danger of overfitting using a GA, these fits can help provide a better understanding of the model and architecture, including how the architecture changes under moderators such stress. This paper also includes some notes on model maintenance and reporting.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8
Fig. 9
Fig. 10
Fig. 11
Fig. 12

Similar content being viewed by others

Notes

  1. Just to be clear, this seems like a horrible tax to pay.

References

  • Anderson JR (2000) Learning and memory: an integrated approach. John Wiley and Sons, New York, NY

    Google Scholar 

  • Anderson JR (2007) How can the human mind exist in the physical universe? Oxford University Press, New York, NY

    Google Scholar 

  • Anderson JR (2014) Cognitive psychology and its implications, 8th edn. Worth Publishers, New York, NY

    Google Scholar 

  • Anderson JR, Bothell D, Byrne MD, Douglass S, Lebiere C, Qin Y (2004) An integrated theory of the mind. Psychol Rev 111(4):1036–1060

    Google Scholar 

  • Bass EJ, Baxter GD, Ritter FE (1995) Creating models to control simulations: a generic approach. AI Simul Behav Q 93:18–25

    Google Scholar 

  • Brener M, Becera N, Pruett D, Ritter FE, Dancy CL, Webster I. (2019) Manual for HumMod (Salt Version 3.0.4) (Tech. Report No. ACS 2019–1, Version: 2.9) Applied Cognitive Science Lab, College of Information Sciences and Technology. Penn State Univ acs.ist.psu.edu/reports/brenerBBPRDW19.pdf

  • Busetta P, Howden N, Rönnquist R, Hodgson A (1999a) Structuring BDI agents in functional clusters. In Proceedings of the Sixth International Workshop on Agent Technologies, Architectures and Languages 149–161.

  • Busetta P, Rönnquist R, Hodgson A, Lucas A (1999b) JACK intelligent agents—Components for intelligent agents in JAVA. AgentLink News Letter, 2(Jan.) www.agent-software.com/white-paper.pdf.

  • Byrne MD, Anderson JR (1998) Perception and action. In: Anderson JR, Lebiere C (eds) The atomic components of thought. Erlbaum, Mahwah, NJ

    Google Scholar 

  • Colbert S (2007) I am America (and so can you!) New York. Grand Central Publishing Hachette Book Group, NY

    Google Scholar 

  • Cornwell JB (2001) Using genetic algorithms to create and optimize cognitive models of development. Unpublished BS honors thesis, The Pennsylvania State University, University Park.

  • Dancy CL (2013) ACT-RΦ: a cognitive architecture with physiology and affect. Biol Inspir Cogn Archit 6:40–45

    Google Scholar 

  • Dancy CL (2019) A hybrid cognitive architecture with primal affect and physiology. IEEE Trans Affect Comput. https://doi.org/10.1109/TAFFC.2019.2906162

    Article  Google Scholar 

  • Dancy CL Kaulakis R (2013) Towards adding bottom-up homeostatic affect to ACT-R. In Proceedings of the 12th International Conference on Cognitive Modeling, 316–321. Ottawa, Canada.

  • Dancy CL Kim JW (2018) Towards a physio-cognitive model of slow-breathing. In Proceedings of the 40th Annual Conference of the Cognitive Science Society, 1587–1592. Madison, WI.

  • Dancy CL, Ritter FE (2017a) IGT-Open: an open source, computerized version of the Iowa Gambling Task. Behav Res Methods 49(3):972–978

    Google Scholar 

  • Dancy CL, Ritter FE (2017b) A standard model of the mind needs a body. In Proceedings of the AAAI Fall Symposium Series, 316–320. Arlington, VA.

  • Dancy CL, Schwartz DM (2017) A computational cognitive-affective model of decision-making. In Proceedings of the 15th International Conference on Cognitive Modeling, 31–36. Coventry, United Kingdom: University of Warwick.

  • Dancy CL Ritter FE, Berry K (2012) Towards adding a physiological substrate to ACT-R. In Proceedings of the 21st Conference on Behavior Representation in Modeling and Simulation, 12-BRIMS-014, 078–085. Amelia Island, FL: BRIMS Society.

  • Dancy CL, Ritter FE, Berry K, Klein LC (2015a) Using a cognitive architecture with a physiological substrate to represent effects of psychological stress on cognition. Comput Math Organ Theory 21(1):90–114

    Google Scholar 

  • Dancy CL, Ritter FE, Gunzelmann G (2015b) Two ways to model the effects of sleep fatigue on cognition. In: Proceedings of the 13th international conference on cognitive modeling 2015, pp 258–263

  • Davis LW, Ritter F (1987) Schedule optimization with probabilistic search. In The Third IEEE Computer Society Conference on Artificial Intelligence Applications, 231–236. Washington, DC: IEEE Press.

  • Evertsz R, Pedrotti M, Busetta P, Acar H, Ritter FE (2009) Populating VBS2 with realistic virtual actors. In Proceedings of the 18th Conference on Behavior Representation in Modeling and Simulation, 09-BRIMS-04.

  • Findlay JM, Gilchrist ID (2003) Active Vision: the psychology of looking and seeing. Oxford University Press, Oxford, UK

    Google Scholar 

  • Gemma E, Helm R, Johnson R, Vlissides J (1995) Design patterns: elements of reusable object-oriented software. Addison-Wesley, Boston, MA

    Google Scholar 

  • Gluck KA, Pew RW (eds) (2005) Modeling human behavior with integrated cognitive architectures: comparison, evaluation, and validation. Erlbaum, Mahwah, NJ

    Google Scholar 

  • Goldberg DE (1989) Genetic algorithms in search, optimization, and machine learning. Addison-Wesley, Reading, MA

    Google Scholar 

  • Gratch J, Marsella S (2004) A domain-independent framework for modeling emotion. Cogn Sys Res 5(4):269–306

    Google Scholar 

  • Gray WD (2000) The nature and processing of errors in interactive behavior. Cogn Sci 24(2):205–248

    Google Scholar 

  • Gray WD (2002) Simulated task environments: the role of high-fidelity simulations, scaled worlds, synthetic environments, and microworlds in basic and applied cognitive research. Cogn Sci Quart 2(2):205–227

    Google Scholar 

  • Gray WD (2007) Composition and control of integrated cognitive systems In WD Gray Ed Integrated models of cognitive systems (pp. 3–12). New York, NY, Oxford University Press.

  • Gray WD (2008) Cognitive architectures: choreographing the dance of mental operations with the task environment. Hum Factors 50(3):497–505

    Google Scholar 

  • Grossman D (1996) on killing: the psychological cost of learning to kill in war and society. Back Bay Books, Little Brown and Company, New York, NY

    Google Scholar 

  • Hall JE (2016) Guyton and Hall textbook of medical physiology, 13th edn. Elsevier, Philadelphia, PA

    Google Scholar 

  • Hester RL, Brown AJ, Husband L, Iliescu R, Pruett D, Summers R et al (2011) HumMod: a modeling environment for the simulation of integrative human physiology. Front Physiol 2(12):Article 12

    Google Scholar 

  • Hope RM, Schoelles MJ, Gray WD (2014) Simplifying the interaction between cognitive models and task environments with the JSON network interface. Behav Res Methods 46(4):1007–1012

    Google Scholar 

  • Jones G, Ritter FE, Wood DJ (2000) Using a cognitive architecture to examine what develops. Psychol Sci 11(2):93–100

    Google Scholar 

  • Kase SE (2008) Parallel genetic algorithm optimization of a cognitive model: Investigating group and individual performance on a math stressor task. Unpublished PhD thesis, College of IST, Penn State University, University Park, PA.

  • Kase SE, Ritter FE, Bennett JM, Klein LC, Schoelles M (2017) Fitting a model to behavior reveals what changes cognitively when under stress and with caffeine. Biol Ins Cogn Architec 22:1–9

    Google Scholar 

  • Kim JW, Dancy CL, Sottilare RA (2018) Towards using a physio-cognitive model in tutoring for psychomotor tasks. In Proceedings of the AIED Workshop on Authoring and Tutoring for Psychomotor, Mobile, and Medical Domains. London, UK.

  • Kim JW, Ritter FE (2015) Learning, forgetting, and relearning for keystroke- and mouse-driven tasks: relearning is important. Human-Computer Interaction 30(1):1–33

    Google Scholar 

  • Kirschbaum C, Pirke K-M, Hellhammer DH (1993) The Trier Social Stress Test—a tool for investigating psychobiological stress responses in a laboratory setting. Neuropsychobiology 28:76–81

    Google Scholar 

  • Laird JE (2012) The Soar cognitive architecture. MIT Press, Cambridge, MA

    Google Scholar 

  • Lane PCR, Gobet F (2005) Discovering predictive variables when evolving cognitive models. In: Pattern recognition and data mining. Third International Conference on Advances in Pattern Recognition, ICAPR 2005, Bath, UK, August 22–25, 2005, Proceedings, Part I. Springer, Heidelberg, Germany, pp 108–117

  • Larue O, West RL, Rosenbloom PS, Dancy CL, Samsonovich AV, Petters D et al (2018) Emotion in the Common Model of Cognition. Procedia Computer Science 145:740–746

    Google Scholar 

  • Lazarus RS, Folkman S (1984) Stress, appraisal and coping. Springer Publishing, New York, NY

    Google Scholar 

  • Moore LR Jr (2011) Cognitive model exploration and optimization: A new challenge for computational science. Comput Math Org Theory 17:296–313

    Google Scholar 

  • Morgan GP, Ritter FE, Stine MM, Klein LC (2006) The cognitive effects of caffeine: implications for models of users: ACS Lab, College of IST, Penn State. Unpublished manuscript.

  • Morgan JH, Morgan GP, Ritter FE (2010) A preliminary model of participation for small groups. Comput Math Org Theory 16:246–270

    Google Scholar 

  • Morgan JH, Morgan GP, Ritter FE. Poncelin de Raucourt V (2009) A preliminary model of participation. In Proceedings of the 18th Conference on Behavior Representation in Modeling and Simulation, 129–136. 109-BRIMS-127.

  • Newell A (1990) Unified theories of cognition. Harvard University Press, Cambridge, MA

    Google Scholar 

  • Norling E, Ritter FE (2004) CoJACK Software Specification. Human variability within computer generated forces project, Project No: RT/COM/3/006: Agent Oriented Software Limited.

  • Ong R (1994) Mechanisms for routinely tying cognitive models to interactive simulations. Unpublished MSc thesis, U. of Nottingham. Available as ESRC Centre for Research in Development, Instruction and Training Technical report #21 and as https://acs.ist.psu.edu/misc/nottingham/ccc/mongsu-2.1.tar.gz.

  • Paik J, Kim JW, Ritter FE, Reitter D (2015) Predicting user performance and learning in human-computer interaction with the Herbal compiler. ACM Trans Comput-Hum Interact 22(5):Article 25

    Google Scholar 

  • Pampel FC (2000) Logistic regression: A primer. Sage, Thousand Oaks, CA

    Google Scholar 

  • Peebles D (2016) Two methods for optimising cognitive model parameters. In: Presentations at the ACT-R Post-graduate Summer School.

  • Pew RW (2007) Some history of human performance modeling. In: Gray W (ed) Integrated models of cognitive systems. Oxford University Press, New York, NY, pp 29–44

    Google Scholar 

  • Pew RW, Mavor AS (eds) (1998) Modeling human and organizational behavior: Application to military simulations. National Academies Press, Washington, DC

    Google Scholar 

  • Pew RW, Mavor AS (eds) (2007) Human-system integration in the system development process: A new look. National Academy Press, Washington, DC

    Google Scholar 

  • Ritter FE (1990) Mendel-DP: Optimizing PDP learning rates. In Eighth Annual Pitt-CMU conference, June and the CMU PDP seminar, June

    Google Scholar 

  • Ritter FE (1991) Towards fair comparisons of connectionist algorithms through automatically generated parameter sets. In: Proceedings of the 13th Annual Conference of the Cognitive Science Society. Erlbaum, Hillsdale, NJ, pp 877–881

  • Ritter FE (1993) Three types of emotional effects that will occur in cognitive architectures. In: Workshop on architectures underlying motivation and emotion (WAUME93). School of Computer Science and Centre for Research in Cognitive Science, University of Birmingham, UK.

  • Ritter FE (2019) Modeling human cognitive behavior for system design. In: Gunther P, Nightingale S, Garcia ACA (eds) Digital human modelling and posturography. Academic Press, London, pp 517–525

    Google Scholar 

  • Ritter FE, Baxter GD, Churchill EF (2014) Foundations for designing user-centered systems: What system designers need to know about people. Springer, London, UK

    Google Scholar 

  • Ritter FE, Baxter GD, Jones G, Young RM (2000) Supporting cognitive models as users. ACM Trans Comput-Hum Interact 7(2):141–173

    Google Scholar 

  • Klein LC, Bennett JM, Whetzel CA, Ritter FE (2008) Daily caffeine use impacts neuroendocrine and cardiovascular responses to laboratory stress in healthy men. Psychosom Med 70(3):A–58

    Google Scholar 

  • Ritter FE, Baxter GD, Jones G, Young RM (2001) User interface evaluation: How cognitive models can help. In: Carroll J (ed) Human-computer interaction in the new millennium. Addison-Wesley, Reading, MA, pp 125–147

    Google Scholar 

  • Ritter FE, Bittner JL, Kase SE, Evertsz R, Pedrotti M, Busetta P (2012) CoJACK: a high-level cognitive architecture with demonstrations of moderators, variability, and implications for situation awareness. Biol Ins Cogn Architec 1(1):2–13

    Google Scholar 

  • Ritter FE, Kase SE, Klein LC, Bennett J, Schoelles M (2009) Fitting a model to behavior tells us what changes cognitively when under stress and with caffeine. In: Proceedings of the biologically inspired cognitive architectures symposium at the AAAI fall symposium. Keynote presentation, Technical Report FS-09-01. AAAI Press, Menlo Park, CA, pp 109–115

  • Ritter FE, Kukreja U, St. Amant R (2007a) Including a model of visual processing with a cognitive architecture to model a simple teleoperation task. J Cogn Eng Decis Mak 1(2):121–147

    Google Scholar 

  • Ritter FE, Reifers AL, Klein LC, Schoelles MJ (2007b) Lessons from defining theories of stress for architectures. In: Gray W (ed) Integrated models of cognitive systems. Oxford University Press, New York, NY, pp 254–262

    Google Scholar 

  • Ritter FE, Schoelles MJ, Klein LC, Kase SE (2007c) Modeling the range of performance on the serial subtraction task. In Proceedings of the 8th International Conference on Cognitive Modeling. Taylor & Francis/Psychology Press, Oxford, UK, pp 299–304

  • Ritter FE, Schoelles MJ, Quigley KS, Klein LC (2011) Determining the number of model runs: treating cognitive models as theories by not sampling their behavior. In: Rothrock L, Narayanan S (eds) Human-in-the-loop simulations: Methods and practice. Springer-Verlag, London, pp 97–116

    Google Scholar 

  • Ritter FE, Shadbolt NR, Elliman D, Young RM, Gobet F, Baxter GD (2003) Techniques for modeling human performance in synthetic environments: a supplementary review. Human Systems Information Analysis Center (HSIAC), Wright-Patterson Air Force Base, OH

    Google Scholar 

  • Ritter FE, Tehranchi F, Oury JD (2019) ACT-R: a cognitive architecture for modeling cognition. Wiley Interdiscip Rev Cogn Sci 10(3):e1488

    Google Scholar 

  • Ritter FE, Van Rooy D, St. Amant R, Simpson K (2006) Providing user models direct access to interfaces: an exploratory study of a simple interface with implications for HRI and HCI. IEEE Trans Sys Man Cybern Part A Sys Hum 36(3):592–601

    Google Scholar 

  • Ritter FE, Yeh KC (2011) Modeling pharmacokinetics and pharmacodynamics on a mobile device to help caffeine users. In: Augmented Cognition International Conference 2011, FAC 2011, HCII 2011, LNAI 6780. Springer-Verlag, Berlin Heidelberg, pp 528–535

  • Salt L, Wise J, Sennersten C, Lindley CA (2016) REACT-R and Unity integration. In: Proceedings on the International Conference on Artificial Intelligence (ICAI), pp 31–37

  • Salvucci DD (2006) Modeling driver behavior in a cognitive architecture. Hum Factors 48:362–380

    Google Scholar 

  • Salvucci DD (2009) Rapid prototyping and evaluation of in-vehicle interfaces. ACM Trans Comput-Hum Interact 16(2):Article 9

    Google Scholar 

  • Salvucci DD (2013) Integration and reuse in cognitive skill acquisition. Cogn Sci 37(5):829–860

    Google Scholar 

  • Shakir A (2002) Assessment of models of human decision-making for air combat analysis. [Unpublished technical report. DERA Farnborough. Abstract, put on the web with permission, at https://acs.ist.psu.edu/papers/shakir02-abstract.pdf].

  • St. Amant R (2000) Interface agents as surrogate users. intelligence 11(2):28–38

    Google Scholar 

  • St. Amant R, Riedl MO (2001) A perception/action substrate for cognitive modeling in HCI. Inter J Hum-Comput Stud 55(1):15–39

    Google Scholar 

  • St. Amant R, Riedl MO, Ritter FE. Reifers A (2005) Image processing in cognitive models with SegMan. In Proceedings of HCI International '05, Volume 4 - Theories Models and Processes in HCI. Paper # 1869. Mahwah, NJ: Erlbaum.

  • St. Amant R, Horton TE, Ritter FE (2007) Model-based evaluation of expert cell phone menu interaction. ACM Trans Comput-Hum Interact 14(1):24

    Google Scholar 

  • Taatgen NA (2002) A model of individual differences in skill acquisition in the Kanfer-Ackerman Air Traffic Control Task. Cogn Sys Res 3(1):103–112

    Google Scholar 

  • Tambe M, Johnson WL, Jones RM, Koss F, Laird JE, Rosenbloom PS et al (1995) Intelligent agents for interactive simulation environments. AI Magazine 16(1):15–40

    Google Scholar 

  • Tehranchi F, Ritter FE 2018. Modeling visual search in interactive graphic interfaces: Adding visual pattern matching algorithms to ACT-R. In: Proceedings of the 16th International Conference on Cognitive Modeling (ICCM 2018). Madison, WI, pp 162–167

  • Tor K, Ritter, FE 2004 Using a genetic algorithm to optimize the fit of cognitive models. In: Proceedings of the Sixth International Conference on Cognitive Modeling. Erlbaum, Mahwah, NJ, 308–313

  • Wallach DP, Fackert S, Albach V (2019) Predictive prototyping for real-world applications: A model-based evaluation approach based on the ACT-R cognitive architecture. In: DIS '19: Proceedings of the 2019 on Designing Interactive Systems Conference, pp 1495–1502

Download references

Acknowledgements

Stephen Colbert’s (2007) I am America (and so can you!) provided inspiration for the title. This report was supported by ONR (N00014-15–1-2275). It is based on a plenary presented at BRIMS 2018. The work reported has been supported by a wide range of sponsors noted in the individual reports. Ritter would like to thank his collaborators, including the ACS Lab, Agent Oriented Systems (Lucas, Evertsz, Pedrotti), Jeanette Bennett, Jen Bittner, Robert Hester, Laura Klein, Drew Pruett, Robert St. Amant, Mike Schoelles, Courtney Whetzel, and folks at Charles River Analytics (Weyhrauch, Niehaus, Lynn). This report was improved by comments from Cesar Colchado, Joseph DiPalma, Raphael Rodriguez, David Schwartz, and two kind, helpful, anonymous reviewers. Steve Crocker provided particularly helpful comments on what we thought was a clean manuscript. Any errors, of course, remain the fault of the authors.

Author information

Authors and Affiliations

  1. Applied Cognitive Science Lab, College of IST, Penn State/University Park, University Park, PA, 16802, USA

    Frank E. Ritter & Farnaz Tehranchi

  2. Department of Computer Science, Bucknell University, Lewisburg, PA, 17837, USA

    Christopher L. Dancy

  3. US Army Research Laboratory, Aberdeen Proving Ground, 21005, USA

    Sue E. Kase

  4. Computer Science and Engineering Department, Penn State/University Park, University Park, PA, 16802, USA

    Farnaz Tehranchi

Authors
  1. Frank E. Ritter
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Farnaz Tehranchi
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Christopher L. Dancy
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Sue E. Kase
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Frank E. Ritter.

Additional information

Publisher's Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Colbert (2007) provided inspiration for the title.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Ritter, F.E., Tehranchi, F., Dancy, C.L. et al. Some futures for cognitive modeling and architectures: design patterns for including better interaction with the world, moderators, and improved model to data fits (and so can you). Comput Math Organ Theory 26, 278–306 (2020). https://doi.org/10.1007/s10588-020-09308-7

Download citation

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s10588-020-09308-7

Keywords

Search

Navigation