Skip to main content
SpringerLink
Log in

The Simplicity and Power model for inductive inference

  • Published:
Artificial Intelligence Review Aims and scope Submit manuscript

Abstract

With this paper we wish to present a simplicity (informally ‘simple explanations are the best’) formalism that is easily and directly applicable to modeling problems in cognitive science. While simplicity has been extensively advocated as a psychologically relevant principle, a general modeling formalism has been lacking. The Simplicity and Power model (SP) is a particular simplicity-based framework, that has been supported in machine learning (Wolff, Unifying computing and cognition: the SP theory and its applications, 2006). We propose its utility in cognitive modeling. For illustration, we provide SP demonstrations of the trade-off between encoding with whole exemplars versus parts of stimuli in learning and the effect of wide versus narrow distributions in categorization. In both cases, SP computations show how simplicity can account for these contrasts, in terms of how the frequency of individual exemplars in training compares to the frequency of their constituent parts.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Similar content being viewed by others

References

  • Anderson JR, Bothell D, Byrne MD, Douglass S, Lebiere C and Qin Y (2004). An integrated theory of the mind. Psychol Rev 111: 1036–1060

    Article  Google Scholar 

  • Baxter R, Oliver J (1994) MDL and MML: similarities and differences. Tech Report 207, Department of Computer Science, Monash University

  • Boucher L and Dienes Z (2003). Two ways of learning associations. Cogn Sci 27: 807–842

    Article  Google Scholar 

  • Chater N (1996). Reconciling simplicity and likelihood principles in perceptual organization. Psychol Rev 103: 566–591

    Article  Google Scholar 

  • Chater N (1997) Simplicity and the mind. The psychologist. November 1997:495–498

  • Chater N (1999). The search for simplicity: a fundamental cognitive principle?. Quart J Exp Psychol 52A: 273–302

    Article  Google Scholar 

  • Chater N, Hahn U (1997) Representational distortion, similarity and the Universal Law of generalization. In: Proceedings of the similarity and categorization workshop 97. University of Edinburgh, pp 31–36

  • Chater N and Manning CD (2006). Probabilistic models of language processing and acquisition. Trends Cogn Sci 10: 335–344

    Article  Google Scholar 

  • Chater N and Oaksford M (1999). The probability heuristics model of syllogistic reasoning. Cogn Psychol 38: 191–258

    Article  Google Scholar 

  • Feldman J (2004). How surprising is a simple pattern? Quantifying “Eureka!”. Cognition 93: 199–224

    Article  Google Scholar 

  • Garner WR (1974). The processing of information and structure. LEA, Potomac, Md

    Google Scholar 

  • Grunwald PD, Myung J, Pitt MA (eds) (2005). Advances in minimum description length: theory and applications. Cambridge, MIT Press

    Google Scholar 

  • Hahn U and Chater N (1998). Similarity and rules: distinct? exhaustive? empirically distinguishable?. Cognition 65: 197–230

    Article  Google Scholar 

  • Hahn U, Bailey TM and Elvin LBC (2005). Effects of category diversity on learning, memory, and generalization. Mem Cogn 33: 289–302

    Google Scholar 

  • Hines P, Pothos EM, Chater N (in press) A non-parametric approach to simplicity clustering. Appl Artif Intell

  • Hochberg JE and McAlister E (1953). A quantitative approach to figural goodness. J Exp Psychol 46: 361–364

    Article  Google Scholar 

  • Howson C and Urbach P (1993). Scientific reasoning: the Bayesian approach. Open Court, Chicago

    Google Scholar 

  • Knowlton BJ and Squire LR (1996). Artificial grammar learning depends on implicit acquisition of both abstract and exemplar-specific information. J Exp Psychol: Learn, Mem Cogn 22: 169–181

    Article  Google Scholar 

  • LiM Vitanyi P (1997). An introduction to Kolmogorov complexity and its applications, 2nd edn. Springer-Verlag, Berlin

    Google Scholar 

  • Logan GD (1988). Toward an instance theory of automatization. Psychol Rev 95: 492–527

    Article  Google Scholar 

  • Love BC, Medin DL and Gureckis TM (2004). SUSTAIN: a network model of category learning. Psychol Rev 111: 309–332

    Article  Google Scholar 

  • Mareschal D, Quinn PC and French RM (2002). Asymmetric interference in 3- to 4-month-olds’ sequential category learning. Cogn Sci 26: 377–389

    Article  Google Scholar 

  • Marr D (1982). Vision: a computational investigation into the human representation and processing of visual information. W. H. Freeman, San Francisco

    Google Scholar 

  • Meulemans T, van der Linden M (1997). Associative chunk strength in artificial grammar learning. J Exp Psychol: Learn, Mem, Cogn 23: 1007–1028

    Article  Google Scholar 

  • Miller GA (1958). Free recall of redundant strings of letters. J Exp Psychol 56: 485–491

    Article  Google Scholar 

  • Norenzayan A and Heine SJ (2005). Psychological universals: what are they and how can we know?. Psychol Bull 131: 763–784

    Article  Google Scholar 

  • Nosofsky RM (1989). Further tests of an exemplar-similarity approach to relating identification and categorization. J Exp Psychol: Percept Psychophys 45: 279–290

    Google Scholar 

  • Oaksford M and Chater N (1991). Against logicist cognitive science. Mind Lang 6: 1–38

    Google Scholar 

  • Oaksford M and Chater N (1994). A rational analysis of the selection task as optimal data selection. Psychol Rev 101: 608–631

    Article  Google Scholar 

  • Oaksford M, Chater N, Grainger B and Larkin J (1997). Optimal data selection in the reduced array selection task (RAST). J Exp Psychol: Learn, Mem, Cogn 23: 441–458

    Article  Google Scholar 

  • Perruchet P, Vinter A, Pacteau C and Gallego J (2002). The formation of structurally relevant units in artificial grammar learning. Quart J Exp Psychol 55A: 485–503

    Google Scholar 

  • Pomerantz JR and Kubovy M (1986). Theoretical approaches to perceptual organization: simplicity and likelihood principles. In: Boff, KR, Kaufman, L and Thomas, JP (eds) Handbook of perception and human performance. vol. II, pp 1–45. Wiley, Cognitive processes and performance

    Google Scholar 

  • Posner MI and Keele SW (1968). On the genesis of abstract ideas. J Exp Psychol 77: 353–363

    Article  Google Scholar 

  • Pothos EM (2005). The rules versus similarity distinction. Behav Brain Sci 28: 1–49

    Article  Google Scholar 

  • Pothos EM, Bailey TM (1999) An entropy model of artificial grammar learning. In: Proceedings of the twenty-first annual conference of the cognitive science society, LEA, Mahwah, pp 549–554

  • Pothos EM and Bailey TM (2000). The importance of similarity in artificial grammar learning. J Exp Psychol: Learn, Mem, Cogn 26: 847–862

    Article  Google Scholar 

  • Pothos EM and Chater N (2002). A simplicity principle in unsupervised human categorization. Cogn Sci 26: 303–343

    Article  Google Scholar 

  • Pothos EM and Chater N (2005). Unsupervised categorization and category learning. Quart J Exp Psychol 58A: 733–752

    Google Scholar 

  • Pothos EM and Ward R (2000). Symmetry, repetition and figural goodness: an investigation of the weight of evidence theory. Cognition 75: B65–B78

    Article  Google Scholar 

  • Reber AS (1989). Implicit learning and tacit knowledge. J Exp Psychol: General 118: 219–235

    Article  Google Scholar 

  • Regehr G and Brooks LR (1993). Perceptual manifestations of an analytic structure: the priority of holistic individuation. J Exp Psychol: General 122: 92–114

    Article  Google Scholar 

  • Rips LJ (1989). Similarity, typicality and categorization. In: Vosniadou S, Ortony A (eds) Similarity and analogical reasoning. Cambridge University Press, Cambridge

    Google Scholar 

  • Rissanen J (1978). Modeling by shortest data description. Automatica 14: 465–471

    Article  MATH  Google Scholar 

  • Rissanen J (1987). Stochastic complexity. J Royal Stat Soc Ser B 49: 223–239

    MATH  MathSciNet  Google Scholar 

  • Rissanen J (1989). Stochastic complexity and statistical inquiry. World Scientific, Singapore

    Google Scholar 

  • Rouder JN and Ratcliff R (2006). Comparing exemplar- and rule-based theories of categorization. Curr Direction Psychol Sci 15: 9–13

    Article  Google Scholar 

  • Shin HJ and Nosofsky RM (1992). Similarity-scaling studies of “dot-pattern” classification and recognition. J Exp Psychol: General 121: 278–304

    Article  Google Scholar 

  • Smith EE and Sloman SA (1994). Similarity—vs. rule—based categorization. Mem Cogn 22: 377–386

    Google Scholar 

  • Solomonoff RJ (1964) A formal theory of inductive inference. Parts I and II. Inf Control 7:1–22, 224–254

    Google Scholar 

  • Solomonoff RJ (1978). Complexity-based induction systems: comparisons and convergence theorems. IEEE Trans Inf Theory 24: 422–432

    Article  MATH  MathSciNet  Google Scholar 

  • Tenenbaum J and Griffiths TL (2001). Generalization, similarity and Bayesian inference. Behav Brain Sci 24: 629–641

    Article  Google Scholar 

  • Tversky A (1977). Features of similarity. Psychol Rev 84: 327–352

    Article  Google Scholar 

  • Vitanyi PMB, Li M (1997) On prediction by data compression. In: Proceedings of 9th European conference on machine learning, lecture notes in artificial intelligence, vol 1224. Springer-Verlag, Heidelberg, pp 14–30

  • Leeuwenberg LJ and Helm PA (1996). Goodness of visual regularities: A nontransformational approach. Psychol Rev 103: 429–456

    Article  Google Scholar 

  • Vokey JR and Brooks LR (1992). Salience of item knowledge in learning artificial grammar. J Exp Psychol: Learn, Mem Cogn 20: 328–344

    Google Scholar 

  • Wallace CS and Boulton DM (1968). An information measure for classification. Comp J 11: 185–195

    MATH  Google Scholar 

  • Wallace CS and Freeman PR (1987). Estimation and inference by compact coding. J R Stat Soc, Ser B 49: 240–251

    MATH  MathSciNet  Google Scholar 

  • Wasserman EA and Miller RR (1997). What’s elementary about associative learning?. Ann Rev Psychol 48: 573–607

    Article  Google Scholar 

  • Wolff JG (2003). Information compression by multiple alignment, unification and search as a unifying principle in computing and cognition. Artif Intell Rev 19(3): 193–230

    Article  Google Scholar 

  • Wolff JG (2006) Unifying computing and cognition: the SP theory and its applications. CognitionResearch.org.uk, Menai Bridge. ISBN 0–9550726-0-3 (Ebook edition distributed by Amazon.com

Download references

Author information

Authors and Affiliations

  1. Department of Psychology, Swansea University, Swansea, SA2 8PP, UK

    Emmanuel M. Pothos & J. Gerard Wolff

Authors
  1. Emmanuel M. Pothos
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. J. Gerard Wolff
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Emmanuel M. Pothos.

Rights and permissions

Reprints and permissions

About this article

Cite this article

Pothos, E.M., Wolff, J.G. The Simplicity and Power model for inductive inference. Artif Intell Rev 26, 211–225 (2006). https://doi.org/10.1007/s10462-007-9058-x

Download citation

  • Received:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s10462-007-9058-x

Keywords

Search

Navigation