Original articles
Knowledge reduction in formal contexts using non-negative matrix factorization

https://doi.org/10.1016/j.matcom.2014.08.004Get rights and content

Abstract

Formal Concept Analysis (FCA) is a mathematical framework that offers conceptual data analysis and knowledge discovery. One of the main issues of knowledge discovery is knowledge reduction. The objective of this paper is to investigate the knowledge reduction in FCA and propose a method based on Non-Negative Matrix Factorization (NMF) for addressing the issue. Experiments on real world and benchmark datasets offer the evidence for the performance of the proposed method.

Introduction

Introduced by Rudolf Wille in the mid 80s, Formal Concept Analysis (FCA) has brought mathematical thinking for knowledge representation and discovery. FCA is formulated based on the two important notions: formal context and formal concept. A formal context consists of a set of objects, set of attributes and a binary relation specifying which objects have what attributes. A formal context is modeled as a cross table, with rows representing the formal objects, columns representing the formal attributes and the crosses representing the relations between them. Concepts derived from the formal context sorted by the inclusion order derived from the formal context sorted by the inclusion order () forms a complete lattice known as concept lattice. FCA organizes the information through concept lattices which fundamentally comprises partial order, reflecting the relationship of generalization and specialization among the concepts of the context. As an effective tool for data analysis, knowledge representation and processing, FCA has been widely studied and applied in many diverse scientific fields  [1], [42].

Two central issues in FCA based knowledge discovery are the knowledge representation and knowledge reduction. Recently, there is a growing interest among FCA research community on knowledge reduction. Formal contexts of modest size can produce thousands of formal concepts, further resulting in unreadable and unmanageable concept lattices  [28], [8]. Another pertinent issue is the density and noise within a context that increase the number of formal concepts. Hence the basic problem in FCA is to find minimal contextual structure which avoids the redundancy while maintaining the structure consistency. In several practical applications the data is of high dimensional in nature and hence often it is required to uncover its low dimensional structure. Matrix decomposition techniques are proved to be successful in this task. However, while revealing the low dimensionality, it is necessary to preserve the non-negative character of the data. NMF has received significant attention from research communities due to its character of preserving the non-negative property of data. NMF is a multivariate data analysis method that presents original data matrix into the product of basis and encoding matrices with non-negative restrictions. Extending upon the analysis of Snasel et al.  [49] and Cherukuri  [7] the present paper explore the NMF based matrix decomposition for knowledge reduction in formal context. Rest of the paper is presented as follows. To facilitate our discussion and make the paper self-contained, basic concepts in FCA are first introduced in Section  2 and further the knowledge reduction in formal contexts with its related work is described. Non-negative matrix factorization is illustrated in Section  3. To reduce the formal context, we introduce a new method based on NMF in Section  4. We demonstrate the experiments using the proposed method and present the analysis of the results in Section  5.

Section snippets

Background

This section recalls the notions and terminology of FCA. Also we summarize the available research on knowledge reduction in FCA. The mathematical principles for this domain were established by Birkhoff in 1960’s and the current form of FCA framework was introduced by Wille in 1980’s.

Knowledge reduction in formal contexts

It is discussed in the literature that generating all formal concepts from a large size formal context and further arranging these concepts hierarchically, exhibits an exponential behavior  [10], [28]. In practice, the computational cost of the same can be prohibitive in many applications. The number of nodes in a concept lattice has linear complexity with size of the formal context. Hence the goal is to minimize the input data before construction of concept lattices and implication bases.

Non-negative matrix factorization

The purpose of this section is to shed some light on NMF. This matrix factorization scheme was initially proposed by Lee and Seung  [31] in an effort to preserve much of the structure of original data while data reduction and also guarantee that both basis and weights are non-negative. NMF belongs to a category of unsupervised matrix decompositions whose members share the property that they are designed for the datasets in which attribute values are never negative. This non-negativity

Knowledge reduction using NMF

An analogy can be brought between the formal contexts and the Boolean matrices by replacing the crosses in formal contexts by 1s and blanks by 0s and vice-versa. Further this analogy will allow us to apply the matrix decomposition techniques for knowledge reduction. The main idea is to equate the dimensionality reduction of binary incidence matrix using NMF with knowledge reduction in formal context. Following is the proposed method:

  • 1.

    From the given formal context, construct the binary

Experimental results

We report here the experimental results of the proposed method on two healthcare datasets and datasets obtained from UCI machine learning repository. The healthcare datasets are the part of consumer healthcare informatics project of Medical Research Council of South Africa  [22]. The diseases which were studied in the project are Tuberculosis (TB), Chronic Bronchitis (CB) and Hypertension (HP). However, in our analysis we consider only TB and HP diseases due to the fact that CB dataset does not

Conclusions

Exploiting the relation between matrix decompositions and concept lattice connections, in this paper we have proposed NMF based method for knowledge reduction in FCA. The proposed methodology is evaluated with real world and benchmark datasets. In brief, the main observations are summarized as follows:

  • Formal context reduction using the proposed NMF based method is computationally advantageous than the SVD based method.

  • Knowledge derived from NMF–FCA (k=6, t=0.5; k=3, t=0.8) based reduced TB

Acknowledgments

The first author sincerely acknowledges the financial support from National Board of Higher Mathematics, Dept. of Atomic Energy, Govt. of India under the grant number 2/48(11)/2010-R&D II/10806. The second author would like to thank the partial financial support of the Federal Service of Data Processing (SERPRO).

References (60)

  • J.-S. Mi et al.

    Approaches to attribute reduction in concept lattices induced by axialities

    Knowl.-Based Syst.

    (2010)
  • P.E. Pattison et al.

    Lattices and dimensional representations: matrix decompositions and ordering structures

    Social Networks

    (2002)
  • J. Poelmans et al.

    Formal concept analysis in knowledge processing: A survey on models and techniques

    Expert Systems with Applications

    (2013)
  • M.-W. Shao et al.

    Rule acquisition and complexity reduction in formal decision contexts

    Int. J. Approx. Reason.

    (2014)
  • L. Wang et al.

    A new algebraic structure for formal concept analysis

    Inform. Sci.

    (2010)
  • X. Wang et al.

    Relations of attribute reduction between object and property oriented concept lattices

    Knowl.-Based Syst.

    (2008)
  • H. Wang et al.

    Approaches to knowledge reduction in generalized consistent decision formal context

    Math. Comput. Model.

    (2008)
  • L. Wei et al.

    Relation between concept lattice reduction and rough set reduction

    Knowl.-Based Syst.

    (2010)
  • R. Wille

    Concept lattices and conceptual knowledge systems

    Comput. Math. Appl.

    (1992)
  • R. Belohlavek et al.

    Selecting important concepts using weights

  • R. Belohlavek et al.

    Formal concept analysis with background knowledge: Attribute priorities

    IEEE Trans. Syst. Man Cybern. C

    (2009)
  • R. Belohlavek et al.

    Factor analysis of incidence data via novel decomposition of matrices

  • Ch. Aswani Kumar

    Mining association rules using non-negative matrix factorization and formal concept analysis

  • K. Cheung et al.

    Complexity reduction in lattice-based information retrieval

    Information Retrieval

    (2005)
  • V. Codocedo, C. Taramasco, H. Astudillo, et al. Cheating to achieve formal concept analysis over a large formal...
  • B. Davey et al.

    Introduction to Lattices and Order

    (1990)
  • S.M. Dias, Algoritmos para geração de reticulados conceituais (Master’s thesis) Federal University of Minas Gerais...
  • S.M. Dias, N.J. Vieira, Reducing the size of concept lattices: The JBOS approach, in: Proceedings of the 7th...
  • R. Divya et al.

    Deceiving communication links on an organization email corpus

    J. Comput. Sci.

    (2011)
  • Y. Djouadi et al.

    Possibility theory and formal concept analysis: Context decomposition and uncertainty handling

  • Cited by (86)

    • Knowledge reduction methods of covering approximate spaces based on concept lattice

      2020, Knowledge-Based Systems
      Citation Excerpt :

      The objective of reduction in rough sets is to reduce the redundant knowledge and keep the required properties unchanged. Many reduction methods and algorithms have been developed in various knowledge systems [16,39–44]. In terms of concept lattices, knowledge reduction is the search for a minimal attribute subset that preserves original hierarchical structure of lattices by deleting redundant attributes from formal contexts.

    • Complex neutrosophic δ-equal concepts and their applications in water quality

      2020, Data Analytics in Biomedical Engineering and Healthcare
    View all citing articles on Scopus

    This paper is an extended version of “Mining association rules using non-negative matrix factorization and formal concept analysis” In Proceedings of ICIP 2012, Springer, CCIS 157, pp. 31–39.

    View full text