Cluster versus grid for operational generation of ATCOR’s modtran-based look up tables
Introduction
The ancestor of the modtran line of atmospheric modelling software was first released in 1972, based on band models developed in the 1950’s and 1960’s used to describe atmospheric transmission and absorption behaviour [1]. Over time, not only has the software has been continuously updated [2], [3], but also the underlying molecular spectroscopic database [4]. Although competing models also exist, modtran has become established as a de facto standard in fields related to atmospheric physics and remote sensing. While many applications employing modtran have usage patterns that require only tens of executions, others require hundreds or thousands. Examples of this usage include sensitivity studies [5], [6], and the radiometric and spectral calibration of imaging spectrometers using known characteristics of solar and atmospheric absorption features [7], [8], [9].
When modtran became more widely-used in operational, remotely sensed spectroscopy, invasive modifications of the typically end-user-compiled Fortran source code were developed by a group of users in order to introduce single-run parallelism, which resulted in efficient and scalable speedups [10]. For whatever reason, these code modifications were not officially incorporated into the standard modtran release, which has grown to over 80,000 lines of code. The code base has moved on, producing multiple releases since then, and these parallel modifications have effectively been lost to the wider scientific community.
Meanwhile, further standard applications, including atmospheric correction software such as employed by ISDAS [11] and the widely-disseminated atcor family of software [12], [13] were being built using modtran calculations as the basis of their own algorithms. Atmospheric correction of multispectral/hyperspectral imagery involve calculations that depend on a number of varying modtran-modifiable parameters defining atmospheric conditions (e.g., molecular absorber concentrations, aerosol scattering, optical depth) as well as observer and solar geometry, i.e., flight altitude, heading, ground elevation, view and solar zenith and azimuth angles. Therefore a large number of modtran calls are required to process a single satellite or airborne scene. In software such as isdas [14] and atcor [15], these computations are usually performed off-line, and stored in luts prior to actual atmospheric correction of a scene to enable reasonable operational hyperspectral data processing times. The use of luts support the processing of a large variety of unrelated scenes in a much shorter amount of time than would be needed for direct computation.
In the case of atcor, researchers from multiple institutions began to collaborate, in an informal way, on extending functionality into novel special-purpose areas, ranging from the haze removal of low-spectral, high-spatial IKONOS imagery to the processing of hyperspectral, wide field-of-view (fov) imagery as obtained from airborne (as opposed to space-bourne) instruments. This increased collaboration also increased the desire for higher turnaround on lut generation and, in turn, created the opportunity to pool the collaborators computing resources, in an informal way, to speed up lut generation.
In the lut generation usage of modtran, individual executions are independent of each other and input data parameters can be pre-computed ahead of time, leading to a workload that is “embarrassingly parallel” – i.e. there is no particular effort needed to segment the problem into a large number of parallel tasks. It is a common pattern in the employment of parallel computing for embarrassingly parallel problems to run multiple instances of a particular program over varying input parameters within a given problem space. This usage is explored here in both grid and cluster processing environments.
Section snippets
Method
The stated goals of this study include: the parallel execution of multiple instances of opaque third-party software, the consistency of numeric results in light of a heterogeneous compute environment, the ability to make use of otherwise idle computing resources available to researchers located at multiple institutions, and acceptable total turnaround time. These goals in addition to the parallel decomposition strategy are addressed individually.
Results
Numerical differences of the gnu-compiled modtran-generated results on the four different platforms are characterized in Fig. 5 by mean relative deviations [25] of a per-case “profile” across all 45,056 cases. The “profile” for a case, which is only used for comparison of that case with the other platforms, is defined as the mean of the subset of only those component radiance values that are eventually recorded in the resulting lut. All cases which produced bad i.e., negative, results (see
High-level granularity of modtran
Although it cannot be known what performance effect would have resulted from applying parallelization at a lower level of granularity i.e., within modtran itself, according to Table 1, a sufficiently fast turnaround time is clearly achievable using either of the aforementioned grid or cluster approaches on current hardware. With continuous hardware improvements, this situation is only expected to improve.
Numeric consistency evaluation
The most important information depicted in Fig. 5 is that other than the large number of
Conclusions
The application of grid and cluster parallelization in executing thousands of runs of third party software with varying input parameters has been investigated, specifically for the generation of atcor’s lut, widely-used in earth observation applications. The importance of validating the numerical consistency of the results for numerically intensive scientific software such as modtran, especially in light of heterogeneous computing environments, has been shown. In this case study, the potential
Acknowledgements
This work was supported in part by ESA/ESTEC contracts 16298/02/NL/US and 15449/01/NL/Sfe. The authors wish to acknowledge the support of the University of Zurich IT Services Department for access to the Matterhorn cluster [26]. J. Brazile acknowledges the support of Netcetera AG throughout the period of this work.
References (39)
- et al.
modtran cloud and multiple scattering upgrades with applications to AVIRIS
Remote Sensing of Environment
(1998) - et al.
The HITRAN molecular spectroscopic database: edition of 2000 including updates through 2001
Journal of Quantitative Spectroscopy and Radiative Transfer
(2003) - et al.
MODTRAN on supercomputers and parallel computers
Parallel Computing
(2002) - G.P. Anderson, J. Wang, M.L. Hoke, F.X. Kneizys, J.H. Chetwynd, L.S Rothman Jr., L.M. Kimball, R.A. McClatchey, E.P....
- A. Berk, G.P. Anderson, P.K. Acharya, L.S. Bernstein, L. Muratov, J. Lee, M. Fox, S.M. Adler-Golden, J.H. Chetwynd,...
- et al.
Modeling the noise equivalent radiance requirements of imaging spectrometers based on scientific applications
OSA Journal of Applied Optics
(2002) - D. Schläpfer, J. Nieke, Operational simulation of at sensor radiance sensitivity using the MODO/MODTRAN environment,...
- et al.
On-orbit radiometric and spectral calibration characteristics of EO-1 Hyperion derived with an underflight of AVIRIS and in-situ measurements at Salar de Arizaro, Argentina
IEEE Transactions on Geoscience and Remote Sensing
(2003) - R.A. Neville, L. Sun, K. Staenz, Detection of spectral line curvature in imaging spectrometer data, in: S. Shen, P....
- et al.
Scene-based spectral response function shape discernibility for the APEX imaging spectrometer
IEEE Geoscience and Remote Sensing Letters
(2006)
ISDAS – a system for processing/analyzing hyperspectral data
Canadian Journal of Remote Sensing
Geo-atmospheric processing of airborne imaging spectrometry data. Part 2: atmospheric/topographic correction
International Journal of Remote Sensing
Geo-atmospheric processing of airborne imaging spectrometry data Part 1: parametric orthorectification
International Journal of Remote Sensing
Retrieval of surface reflectance from hyperspectral data using a look-up table approach
Canadian Journal of Remote Sensing
What every computer scientist should know about floating-point arithmetic
ACM Computing Surveys
Consistent uncertainty reduction in modeling nonlinear systems
SIAM Journal on Scientific Computing
Cited by (19)
Advanced radiometry measurements and Earth science applicationswith the Airborne Prism Experiment (APEX)
2015, Remote Sensing of EnvironmentCitation Excerpt :The science grade APEX processor is designed to process large quantities of imaging spectrometer data, including calibration and housekeeping data (Schaepman, Schläpfer, Brazile, & Bojinski, 2002). Its design is based on an iterative prototyping approach and from beginning on includes considerations to build look-up tables for atmospheric compensation (Brazile et al., 2004), as well as optimized processing speed requirements (Brazile, Richter, Schläpfer, Schaepman, & Itten, 2008; Brazile et al., 2005). The science grade APEX processor is developed to integrate calibration data with actual measurements, allowing to reconstruct each geometrical position and each radiance of any given detector element (Fig. 4).
Crop planting and type proportion method for crop acreage estimation of complex agricultural landscapes
2012, International Journal of Applied Earth Observation and GeoinformationCitation Excerpt :Atmospheric corrections, geometric corrections and the soil adjusted vegetation index (SAVI) are processed before classification, as described below and shown in Fig. 4. The MODTRAN algorithm removes the effects of atmospheric components, such as O3, CO2, water, and aerosols (Brazile et al., 2008). The meteorological observation data used in the atmospheric corrections, such as aerosol depth are obtained from the China Meteorological Administration.
Earth system science related imaging spectroscopy-An assessment
2009, Remote Sensing of EnvironmentA High-Performance Infrastructure for Remote Sensing Data Applications Using HPC Paradigms
2022, International Journal of Electrical and Electronics ResearchComparative analysis of atmospheric radiative transfer models using the Atmospheric Look-up table Generator (ALG) toolbox (version 2.0)
2020, Geoscientific Model DevelopmentGradient-based automatic lookup table generator for radiative transfer models
2019, IEEE Transactions on Geoscience and Remote Sensing