Using artificial intelligence planning to automate science image data analysis

https://doi.org/10.1016/S1088-467X(99)00016-5Get rights and content

Abstract

In recent times, improvements in imaging technology have made available an incredible array of information in image format. While powerful and sophisticated image processing software tools are available to prepare and analyze the data, these tools are complex and cumbersome, requiring significant expertise to properly operate. Thus, in order to extract (e.g., mine or analyze) useful information from the data, a user (in our case a scientist) often must possess both significant science and image processing expertise.

This article describes the use of artificial intelligence (AI) planning techniques to represent scientific, image processing and software tool knowledge to automate knowledge discovery and data mining (e.g., science data analysis) of large image databases. In particular, we describe two fielded systems. The Multimission VICAR Planner (MVP) which has been deployed for since 1995 and is currently supporting science product generation for the Galileo mission. MVP has reduced time to fill certain classes of requests from 4 h to 15 min. The Automated SAR Image Processing system (ASIP) was deployed at the Department of Geology at Arizona State University in 1997 to support aeolian science analysis of synthetic aperture radar images. ASIP reduces the number of manual inputs in science product generation by ten-fold.

Introduction

Recent breakthroughs in imaging technology have led to an explosion of available data in image format. However, these advances in imaging technology have brought with them a commensurate increase in the complexity of image processing and analysis technology. When a scientist analyzes newly available image data to discover patterns or to confirm scientific theories, they must perform a complex set of operations. First, before the data can be used it must often be reformatted, cleaned, and many correction steps must be applied. Then, in order to perform the actual data analysis, the user must manage all of the analysis software packages and their requirements on format, required information, etc.

Furthermore, this data analysis process is not a one-shot process (indeed, we describe a specific case of the process outlined in [12]. Typically a scientist will set up some sort of analysis, study the results, and then use the results of this analysis to modify the analysis to improve it. This cycle of repeated analysis may occur many times – thus any reduction in the scientist effort or cycle time can dramatically improve scientist productivity.

Unfortunately, this data preparation and analysis process is both knowledge and labor intensive. Consider the task of producing a mosaic of images of the moon from the Galileo mission (corrected for lighting, transmission errors and camera distortions). Consider also that our end goal is to perform geological analyses i.e., to study the composition of the surface materials on the moon. One technique used to do this is to construct a ratio image – an image whose values are the ratio of the intensity of the response at two different bandwidths (e.g., the ratio of infra-red response and visible green response). In order to correctly produce this science product for analysis, one must have knowledge of a wide range of sources including:

  • the particular science discipline of interest (e.g., atmospheric science, planetary geology),

  • image processing and the image processing libraries available,

  • where and how the images and associated information are stored (e.g., calibration files) and

  • the overall image processing environment (e.g., to know how to link together libraries and pass information from one program to another).

Note the extreme breadth of knowledge required to perform this task – it requires science, image processing, database infrastructure and image processing language and scripting programming knowledge. As a result of the vast amounts and breadth of knowledge required, it takes many years of training and experience to become expert at assisting these analyses.

Automated planning technology offers the potential automate many of these data analysis functions [12, p. 50, 5], thus enabling novice users to utilize the software libraries to mine the data. It also allows users who may be expert in some areas but less knowledgeable in other to use the software tools to mine the data.

The remainder of this article is organized as follows. First, we provide a brief overview of the key elements of AI planning. We then describe two fielded planning systems for science data analysis. We first describe the MVP system which automates elements of image processing for science data analysis for data from the Galileo mission. We then describe the ASIP system which automates elements of image processing for science data analysis of synthetic aperture radar (SAR) images.

The principle contributions of this article are two-fold.

  • First, we identify automated selection and configuration of Knowledge Discovery and Datamining (KDD) software tools as an area where AI planning technology can significantly extend KDD capabilities.

  • Second, we describe two systems demonstrating the viability and impact of AI planning on the KDD process.1

Section snippets

Artificial intelligence planning techniques

We have applied and extended techniques from AI Planning to address the knowledge-based software reconfiguration problem in general [9], and two applications in science data analysis (e.g., data mining) in specific. In order to describe this work, we first provide a brief overview of the key concepts from planning technology.2

Planning technology relies on an encoding of possible actions in the domain. In this

The DPLAN planning algorithm

The DPLAN planning algorithm uses a unique combination of the HTN and operator-based planning techniques discussed above. DPLAN operates by refining a set of input top-level goals into a set of low-level operational goals (e.g., executable actions). Plans are represented by a three-tuple: 〈U,C,S〉, where U is a set of non-operational (or high-level) goals, C is a set of constraints and S is a set of operational-goals. At the end of planning, U should be empty and the goals in S are returned as

The multimission VICAR planner (MVP)

MVP [5] partially automates the generation of image processing procedures from user requests and a knowledge-based model of video image communication and retrieval3 (VICAR) image processing area using AI automated planning techniques. VICAR image processing is an instance in a planning problem where:

  • the planning actions or operational

The automated SAR image processing (ASIP) system

ASIP automates synthetic aperture radar (SAR) image processing based on user request and a knowledge-base model of SAR image processing using AI automated planning techniques [13], [14]. SAR operates simultaneously in multipolarizations and multifrequencies to produce different images consisting of radar backscatter coefficients (s0) through different polarizations at different frequencies. ASIP enables construction of an aerodynamic roughness image/map (z09

Related work

Related work can be broadly classified into: related image processing languages, related automated image processing work, and related AI planning work. In terms of related image processing languages, there are many commercial and academic image processing packages such as IDL, Aoips and Merlyn. Generally, these packages have only limited ability to automatically determine how to use different image processing programs or algorithms based on the problem context (e.g., other image processing

Conclusions

This paper has described knowledge-based reconfiguration of data analysis software using AI planning techniques. This represents an important area where AI planning can significantly enhance KDD processes. As evidence of this potential, we described two fielded planning systems that enhance KDD: the MVP system, which automates image processing to support Galileo image data science analysis; and the ASIP system which automates production of aerodynamic roughness maps to support geological

Acknowledgements

This work was performed by the Jet Propulsion Laboratory, California Institute of Technology, under contract with the National Aeronautics and Space Administration. The authors would also like to acknowledge the contributions of other past and present members of the MVP team: Todd Turco, Christine Ying, Shouyi Hsiao, Darren Mutz, Alex Gray, Joe Nieten and Jean Lorre. The authors would also like to acknowledge other contributors to the ASIP project including Dan Blumberg (ASU), Anita Govindjee,

References (30)

  • D. Blumberg et al.

    Field studies of aerodynamic roughness length

    Journal of Arid Environments

    (1993)
  • V. Clement et al.

    A knowledge-based approach to integration of image processing procedures

    Image Understanding

    (1993)
  • T. Matsuyama

    Expert systems for image processing: knowledge-based composition of image analysis processes

    Computer Vision, Graphics, and Image Processing

    (1989)
  • Capdevielle, P. Dalle, Image processing chain construction by interactive goal specification, in: Proceedings of the...
  • D. Chapman

    Planning for conjunctive goals

    Artificial Intelligence

    (1987)
  • D. Charlebois et al.

    A case-based planner to automate reuse of ES software for analysis of remote sensing data

    International Geoscience and Remote Sensing Symposium (IGARSS)

    (1991)
  • S.A. Chien et al.

    Automating image Processing for scientific data analysis of a large image database

    IEEE Transactions on Pattern Analysis and Machine Intelligence

    (1996)
  • S. Chien et al.

    Making an impact: artificial intelligence at the jet propulsion laboratory

    AI Magazine

    (1997)
  • S. Chien, A. Govindjee, T. Estlin, X. Wang, A. Griesel, R. Hill Jr, Automated generation of tracking plans for a...
  • S. Chien, N. Muscettola, K. Rajan, B. Smith, G. Rabideau, Automated planning and scheduling for goal-based autonomous...
  • S. Chien, F. Fisher, H. Mortensen, E. Lo, R. Greeley, A. Govindjee, T. Estlin, X. Wang, Using artificial intelligence...
  • K. Erol, J. Hendler, D. Nau, UMCP: A sound and complete procedure for hierarchical task network planning, in:...
  • U. Fayyad et al.

    From data mining to knowledge discovery in databases

    AI Magazine

    (1996)
  • F. Fisher, E. Lo, S. Chien, R. Greeley, SAR image processing through artificial intelligence planning, in: Proceedings...
  • F. Fisher, S. Chien, E. Lo, R. Greeley, Using artificial intelligence planning to automate SAR image processing for...
  • Cited by (6)

    View full text