Original articlesKnowledge reduction in formal contexts using non-negative matrix factorization☆
Introduction
Introduced by Rudolf Wille in the mid 80s, Formal Concept Analysis (FCA) has brought mathematical thinking for knowledge representation and discovery. FCA is formulated based on the two important notions: formal context and formal concept. A formal context consists of a set of objects, set of attributes and a binary relation specifying which objects have what attributes. A formal context is modeled as a cross table, with rows representing the formal objects, columns representing the formal attributes and the crosses representing the relations between them. Concepts derived from the formal context sorted by the inclusion order derived from the formal context sorted by the inclusion order () forms a complete lattice known as concept lattice. FCA organizes the information through concept lattices which fundamentally comprises partial order, reflecting the relationship of generalization and specialization among the concepts of the context. As an effective tool for data analysis, knowledge representation and processing, FCA has been widely studied and applied in many diverse scientific fields [1], [42].
Two central issues in FCA based knowledge discovery are the knowledge representation and knowledge reduction. Recently, there is a growing interest among FCA research community on knowledge reduction. Formal contexts of modest size can produce thousands of formal concepts, further resulting in unreadable and unmanageable concept lattices [28], [8]. Another pertinent issue is the density and noise within a context that increase the number of formal concepts. Hence the basic problem in FCA is to find minimal contextual structure which avoids the redundancy while maintaining the structure consistency. In several practical applications the data is of high dimensional in nature and hence often it is required to uncover its low dimensional structure. Matrix decomposition techniques are proved to be successful in this task. However, while revealing the low dimensionality, it is necessary to preserve the non-negative character of the data. NMF has received significant attention from research communities due to its character of preserving the non-negative property of data. NMF is a multivariate data analysis method that presents original data matrix into the product of basis and encoding matrices with non-negative restrictions. Extending upon the analysis of Snasel et al. [49] and Cherukuri [7] the present paper explore the NMF based matrix decomposition for knowledge reduction in formal context. Rest of the paper is presented as follows. To facilitate our discussion and make the paper self-contained, basic concepts in FCA are first introduced in Section 2 and further the knowledge reduction in formal contexts with its related work is described. Non-negative matrix factorization is illustrated in Section 3. To reduce the formal context, we introduce a new method based on NMF in Section 4. We demonstrate the experiments using the proposed method and present the analysis of the results in Section 5.
Section snippets
Background
This section recalls the notions and terminology of FCA. Also we summarize the available research on knowledge reduction in FCA. The mathematical principles for this domain were established by Birkhoff in 1960’s and the current form of FCA framework was introduced by Wille in 1980’s.
Knowledge reduction in formal contexts
It is discussed in the literature that generating all formal concepts from a large size formal context and further arranging these concepts hierarchically, exhibits an exponential behavior [10], [28]. In practice, the computational cost of the same can be prohibitive in many applications. The number of nodes in a concept lattice has linear complexity with size of the formal context. Hence the goal is to minimize the input data before construction of concept lattices and implication bases.
Non-negative matrix factorization
The purpose of this section is to shed some light on NMF. This matrix factorization scheme was initially proposed by Lee and Seung [31] in an effort to preserve much of the structure of original data while data reduction and also guarantee that both basis and weights are non-negative. NMF belongs to a category of unsupervised matrix decompositions whose members share the property that they are designed for the datasets in which attribute values are never negative. This non-negativity
Knowledge reduction using NMF
An analogy can be brought between the formal contexts and the Boolean matrices by replacing the crosses in formal contexts by and blanks by and vice-versa. Further this analogy will allow us to apply the matrix decomposition techniques for knowledge reduction. The main idea is to equate the dimensionality reduction of binary incidence matrix using NMF with knowledge reduction in formal context. Following is the proposed method:
- 1.
From the given formal context, construct the binary
Experimental results
We report here the experimental results of the proposed method on two healthcare datasets and datasets obtained from UCI machine learning repository. The healthcare datasets are the part of consumer healthcare informatics project of Medical Research Council of South Africa [22]. The diseases which were studied in the project are Tuberculosis (TB), Chronic Bronchitis (CB) and Hypertension (HP). However, in our analysis we consider only TB and HP diseases due to the fact that CB dataset does not
Conclusions
Exploiting the relation between matrix decompositions and concept lattice connections, in this paper we have proposed NMF based method for knowledge reduction in FCA. The proposed methodology is evaluated with real world and benchmark datasets. In brief, the main observations are summarized as follows:
- •
Formal context reduction using the proposed NMF based method is computationally advantageous than the SVD based method.
- •
Knowledge derived from NMF–FCA (, ; , ) based reduced TB
Acknowledgments
The first author sincerely acknowledges the financial support from National Board of Higher Mathematics, Dept. of Atomic Energy, Govt. of India under the grant number 2/48(11)/2010-R&D II/10806. The second author would like to thank the partial financial support of the Federal Service of Data Processing (SERPRO).
References (60)
- et al.
Generating a catalog of unanticipated schemas in class hierarchies using formal concept analysis
Inf. Softw. Technol.
(2010) - et al.
Discovery of optimal factors in binary data via a novel method of matrix decomposition
J. Comput. System Sci.
(2010) - et al.
Subtractive clustering for seeding non-negative matrix factorizations
Inform. Sci.
(2014) - et al.
Applying the JBOS reduction method for relevant knowledge extraction
Expert Sys. Appl.
(2013) - et al.
Extracting reducible knowledge from ANN with JBOS and FCANN approaches
Expert Syst. Appl.
(2013) - et al.
A multi-level conceptual data reduction approach based on the Lukasiewicz implication
Inform Sci.
(2004) - et al.
A heuristic knowledge-reduction method for decision formal contexts
Comput. Math. Appl.
(2011) - et al.
Knowledge reduction in decision formal contexts
Knowl.-Based Syst.
(2011) - et al.
Incomplete decision contexts: approximate concept construction, rule acquisition and knowledge reduction
Int. J. Approx. Reason.
(2013) - et al.
Reduction method for concept lattices based on rough set theory and its application
Computers and Mathematics with Applications
(2007)
Approaches to attribute reduction in concept lattices induced by axialities
Knowl.-Based Syst.
Lattices and dimensional representations: matrix decompositions and ordering structures
Social Networks
Formal concept analysis in knowledge processing: A survey on models and techniques
Expert Systems with Applications
Rule acquisition and complexity reduction in formal decision contexts
Int. J. Approx. Reason.
A new algebraic structure for formal concept analysis
Inform. Sci.
Relations of attribute reduction between object and property oriented concept lattices
Knowl.-Based Syst.
Approaches to knowledge reduction in generalized consistent decision formal context
Math. Comput. Model.
Relation between concept lattice reduction and rough set reduction
Knowl.-Based Syst.
Concept lattices and conceptual knowledge systems
Comput. Math. Appl.
Selecting important concepts using weights
Formal concept analysis with background knowledge: Attribute priorities
IEEE Trans. Syst. Man Cybern. C
Factor analysis of incidence data via novel decomposition of matrices
Mining association rules using non-negative matrix factorization and formal concept analysis
Complexity reduction in lattice-based information retrieval
Information Retrieval
Introduction to Lattices and Order
Deceiving communication links on an organization email corpus
J. Comput. Sci.
Possibility theory and formal concept analysis: Context decomposition and uncertainty handling
Cited by (86)
Mining actionable concepts in concept lattice using Interestingness Propagation
2024, Journal of Computational ScienceDescribing hierarchy of concept lattice by using matrix
2021, Information SciencesKnowledge reduction methods of covering approximate spaces based on concept lattice
2020, Knowledge-Based SystemsCitation Excerpt :The objective of reduction in rough sets is to reduce the redundant knowledge and keep the required properties unchanged. Many reduction methods and algorithms have been developed in various knowledge systems [16,39–44]. In terms of concept lattices, knowledge reduction is the search for a minimal attribute subset that preserves original hierarchical structure of lattices by deleting redundant attributes from formal contexts.
Complex neutrosophic δ-equal concepts and their applications in water quality
2020, Data Analytics in Biomedical Engineering and HealthcareA comparative study of decision implication, concept rule and granular rule
2020, Information Sciences
- ☆
This paper is an extended version of “Mining association rules using non-negative matrix factorization and formal concept analysis” In Proceedings of ICIP 2012, Springer, CCIS 157, pp. 31–39.