Abstract
A knowledge-based approach to retrieve medical images by feature and content with spatial and temporal constructs is developed. Selected objects of interest in a medical image (e.g. x-ray, MR image) are segmented, and contours are generated from these objects. Features (e.g. shape, size, texture) and content (e.g. spatial relationships among objects) are extracted and stored in a feature and content database. Knowledge about image features can be expressed as a hierarchical structure called a Type Abstraction Hierarchy (TAH). The high-level nodes in the TAH represent more general concepts than low-level nodes. Thus, traversing along TAH nodes allows approximate matching by feature and content if an exact match is not available. TAHs can be generated automatically by clustering algorithms based on feature values in the databases and hence are scalable to large collections of image features. Further, since TAHs are generated based on user classes and applications, they are context- and user-sensitive.
A knowledge-based semantic image model is proposed which consists of four layers (raw data layer, feature and content layer, schema layer, and knowledge layer) to represent the various aspects of an image objects' characteristics. The model provides a mechanism for accessing and processing spatial, evolutionary, and temporal queries.
A knowledge-based spatial temporal query language (KSTL) has developed which extends ODMG's OQL and supports approximate matching of feature and content, conceptual terms, and temporal logic predicates. Further a visual query language has been developed that accept pointclick-and-drag visual iconic input on the screen which is then translated into KSTL. User models are introduced to provide default parameter values for specifying query conditions.
We have implemented a Knowledge-Based Medical Database System (KMeD) at UCLA, and currently under evaluation by the medical staff. The results from this research should be applicable to other multimedia information systems as well.
This work is supported in part by the National Science Foundation Scientific Database Initiative Grant IRI9116849, and also in part by ARPA contract F30602-94-C-0207
This is a preview of subscription content, log in via an institution to check access.
Preview
Unable to display preview. Download preview PDF.
References
M. Arya, W. Cody, C. Faloutsos, J. Richardson, and A. Toga. QBISM: A prototype 3-D medical image database system. Bulletin of the Technical Committee on Data Engineering, 16(1):38–42, March 1993.
J. R. Bach, S. Paul, and R. Jain. A visual information management system for the interactive retrieval of faces. IEEE Transaction on Knowledge and Data Engineering, Oct 1993.
R. G. G. Cattell, editor. The Object Database Standard: ODMG-93 (Release 1.2). Morgan Kaufmann, 1996.
W. W. Chu and Q. Chen. A structured approach for cooperative query answering. IEEE Transactions on Knowledge and Data Engineering, 6(5), Oct 1994.
W. W. Chu, K. Chiang, C. C. Hsu, and H. Yau. An error-based conceptual clustering method for providing approximate query answers. Communications of ACM, Virtual Extension Edition (http://www.acm.org/cacm/extension), December 1996.
W. W. Chu, A. F. Cardenas, and R. K. Taira. KMeD: A knowledge-based multimedia medical distributed database system. Information Systems, 20(2):75–96, 1995.
F. Cuppens and R. Demolombe. How to recognize interesting topics to provide cooperative answering. Information Systems. 14(2):163–173, 1989.
A. F. Cardenas, I. T. Ieong, R. K. Taira, et al. The knowledgebased object-oriented PICQUERY + language. IEEE Trans. on Knowledge and Data Engineering, 5(4):644–657, August 1993.
W. W. Chu, T. Ieong, and R. Taira. A semantic modeling approach for image retrieval by content. The VLDB Journal. 3(4):445–478, Oct. 1994.
S. K. Chang, C. W. Yan, and D. C. Dimitroff. An intelligent image database system. IEEE Transaction on Software Engineering, May 1988.
J. D. N. Dionisio and A. F. Cardenas. A unified data model for representing multimedia, timeline, and simulation data, 1997. To appear in IEEE Transactions on Knowledge and Data Engineering.
D. Daneels, D. Van Campenhout, et al. Interactive outlineing: An improved approach using active contours. In Image and Video Storage and Retrieval, SPIE, 1993.
Frank. PSQL: Pictorial spatial query language. IEEE Transaction on Software Engineering, May 1988.
S. Gibbs, C. Breiteneder, and D. Tsichritzis. Data modeling of time-based media. In D. Tsichritzis, editor, Visual Objects, pages 1–22. Centre Universitaire d'Informatique, Universitè de Genève, 1993.
W. Grimson. Object Recognition by Computer: the role of Geometric Constraints. MIT Press, MIT, MA, 1990.
C. C. Hsu, W. W. Chu, and R. K. Taira. A knowledge-based approach for retrieving images by content. IEEE Transaction on Knowledge and Data Engineering, August 1996.
H. K. Huang and R. K. Taira. Infrastructure design of a picture archiving and communication system. American Journal of Roentgenology, 158:743–749, 1992.
M. D. Levine. Vision in Man and Machine. McGraw Hill Publishing Co., Inc., 1985.
B. J. Liu, R. K. Taira, J. Shim, and P. Keaton. Automatic segmentation of bones from digital hand radiographs. In Proceedings of the SPIE: Medical Imaging Image Processing, volume 2434, pages 659–669, February 1995.
W. N. Martin and J. K. Aggarwal. Computer analysis of dynamic scenes containing currilinear figures. Pattern Recognition, 11:169–178, 1979.
R. Mehrotra and J. Gary. Similar-shape retrieval in shape data management. IEEE Computer, Sep 1995.
L. Mohan and R. L. Kashyap. A visual query language for graphical interaction with schema-intensive databases. IEEE Transactions on Knowledge and Data Engineering, 5(5):843–858, October 1993.
R. S. Michalski and R. E. Stepp. Learning from observation: Conceptual clustering. In R. S. Michalski, J. G. Carbonell, and T. M. Mitchell, editors, Machine Learning, volume 1. Margan Kaufmann Publishers, Inc., 1983.
Wayne Niblack, R. Barber, et al. The qbic project: Querying images by content using color, texture, and shape. In Storage and Retrieval for Images and Video Databases, SPIE, 1993.
M. Sonka, V. Hlavac, and R. Boyle. Image, Processing, Analysis and Machine Vision, volume 1. Chapman & Hall Computing, 1993.
A. Tansel, J. Clifford, S. Gadia, S. Jajodia, A. Segev, and R. Snodgrass, editors. Temporal Databases. The Benjamin/Cummings Publishing Company, Inc., 1993.
R. Wasserman, R. Acharya, et al. Multimodality tumor delineation via fuzzy fusion and deformable modelling. In Proceedings of the SPIE, Medical Image: Image Processing, 1995.
A. J. Worth, S. Lehar, and D. N. Kennedy. A recurrent cooperative/competitive field for segmentation of magnetic resonance brain images. IEEE Transactions on Knowledge and Data Engineering, 4(2):156–161, April 1992.
Author information
Authors and Affiliations
Editor information
Rights and permissions
Copyright information
© 1997 Springer-Verlag Berlin Heidelberg
About this paper
Cite this paper
Chu, W.W., Cárdenas, A.F., Taira, R.K. (1997). Knowledge-based image retrieval with spatial and temporal constructs (Invited paper). In: Raś, Z.W., Skowron, A. (eds) Foundations of Intelligent Systems. ISMIS 1997. Lecture Notes in Computer Science, vol 1325. Springer, Berlin, Heidelberg. https://doi.org/10.1007/3-540-63614-5_2
Download citation
DOI: https://doi.org/10.1007/3-540-63614-5_2
Published:
Publisher Name: Springer, Berlin, Heidelberg
Print ISBN: 978-3-540-63614-4
Online ISBN: 978-3-540-69612-4
eBook Packages: Springer Book Archive