Elsevier

Computers & Chemical Engineering

Volume 61, 11 February 2014, Pages 175-184
Computers & Chemical Engineering

Direct numerical simulation of catalytic combustion in a multi-channel monolith reactor using personal computers with emerging architectures

https://doi.org/10.1016/j.compchemeng.2013.11.002Get rights and content

Highlights

  • Serial reacting flow code was parallelized using multi-cores and GPUs.

  • OpenMP constructs are multi-threading and was used for surface chemistry.

  • Multi-threading was used for material property calculations and species equations.

  • Linear solvers were accelerated using GPUs.

  • 4–5× speedup was attained for multi-channel catalytic combustor simulation.

Abstract

Computational Fluid Dynamic modeling of full-scale monolithic catalytic reactors has remained elusive due to the extreme computational requirements. While simulation of full-scale catalytic reactors would require domain decomposition based parallelism and use of multiple central processing units, significant performance enhancement can be achieved by fully utilizing the compute resources available within each node in emerging architectures. Here, a serial reacting flow solver was used as a starting point. Performance was enhanced using multi-threading for acceleration of surface chemistry, material properties calculations, and species equation solvers, and using graphical processing units for acceleration of the linear solvers and pre-conditioners. Of the two test cases presented here, the largest test case entails steady-state calculations for catalytic methane–air combustion with 22 reaction steps and 19 species within a 13-channel catalytic monolith reactor discretized using 313,872 control volumes. For this particular test case, a speed-up factor of about 4.5 over serial calculations is noted.

Introduction

Simulation of full-scale monolithic catalytic reactors has remained elusive despite tremendous progress in computational resources over the past several decades. Such simulations, referred to henceforth as direct numerical simulation, necessitate resolution of individual channels within the monolith while still capturing the effect of overall size of the catalytic reactor. This results in a multi-scale problem that requires extreme computational resources (Mazumder, 2007) to solve, as will be discussed shortly. Literature review reveals numerous publications on two and three-dimensional Computational Fluid Dynamic (CFD) modeling of a single channel of the monolith (Canu and Vechhi, 2002, Chatterjee et al., 2001, Deutschmann et al., 2000, Grimm and Mazumder, 2008, Groppi et al., 1995, Holder et al., 2006, Mantzaras et al., 2000, Papadias et al., 1999, Raja et al., 2000, Sallamie and Koshkanab, 2003, Young and Finlayson, 1976; among many others). In most cases, due to lack of better alternatives, the knowledge gained from the simulation of a single channel is extrapolated to the full catalytic reactor. Since the channels are coupled to each other through heat transfer, and individual channels may encounter different mass flow rates, extrapolation of the results of a single channel to the entire reactor is not always accurate (Tischer, Correa, & Deutschmann, 2001). An important design issue that is not addressed by such extrapolations is the impact of scale-up or scale-down of a catalytic reactor, i.e., what happens if the overall size (diameter) of a catalytic reactor is increased or decreased while keeping the channel dimensions unchanged?

In recent years, in realization of the need to address size effects of a catalytic reactor, attempts have been made to model full-scale monolithic catalytic reactors. Broadly, two types of approaches have been used. The first approach is one where the monolith is modeled as a porous medium, as is done traditionally for packed-bed reactors. The second approach is one where a “representative” number of channels within the monolith are modeled, and the results are used as inputs in thermal network-type models that predict the temperature distribution within the full catalytic reactor. These two approaches essentially bypass the computing challenges posed by the direct numerical simulation of catalytic reactors. Pertinent work using these two approaches is discussed next.

Modeling the monolith as a porous medium keeps the computational requirements tractable while still providing valuable information about the overall performance of the device. Kolaczkowski and Serbertsioglu (1996) performed analytical modeling of channel interactions in catalytic combustion reactors. The focus of their study was the effect of monolith material properties on heat dissipation. Shuai and Wang (2004) modeled the monolithic reactor using a two-dimensional model, in which the reactor was modeled as a porous medium and surface reactions were modeled using a two-step mechanism. Chen, Alexio, Williams, Leprince, and Yong (2004) performed three-dimensional CFD modeling of flow and heterogeneous reactions in catalytic reactors. The pressure and velocity fields were calculated by modeling the monolith as a porous medium. The surface reaction model was then superimposed on the fluid flow results. The interaction of surface reactions, heat transfer, and fluid flow was not resolved in this study. Guojian and Song (2005) performed CFD simulation of flow distribution inside and upstream of a catalytic reactor. They used a one-step reaction and the catalytic monolith was modeled as a porous medium. Mazumder and Sengupta (2002) used a sub-grid scale model for full-scale modeling of monolithic reactors. The monolithic reactor was modeled as an anisotropic porous medium and sub-grid scale models were employed to represent the heterogeneous chemical reactions occurring inside the reactor. Detailed chemistry was considered in their study. Most recently, Hayes, Fadic, Mmbaga, and Najafi (2012) conducted two-dimensional computations of a full-scale automotive catalytic reactors using a porous medium approach. Both steady state and transient light-off behavior was predicted using a simple reduced chemistry model.

Notable work that belongs to the second category is the work of Braun et al. (2002), who performed three-dimensional transient simulation of a monolithic catalytic reactor. Two-dimensional detailed simulation of reactive flows in a representative number of channels was coupled with a three-dimensional simulation of heat transfer in the solid structure of the monolith. Recently, Tischer and Deutschmann (2005) extended the work done by Braun et al. (2002) by using the so-called aggregation model to identify the representative channels in a monolith. Two-dimensional, steady-state results of reactive flow in these representative channels were coupled with three-dimensional transient models of energy transport in the solid structure of the monolith. In their study, around 10 channels were chosen as representative channels. A similar approach in which one-dimensional results of 80 representative channels were coupled to a three-dimensional thermal calculation was conducted by Stepanek, Koci, Marek, and Kubicek (2012) for predicting transient behavior of a full-scale automotive three-way catalytic reactor.

While the two approaches, just discussed, represent improvement over modeling a single channel, the accuracy (in the case of the porous medium approach) and the numerical robustness (in the case of loose coupling using network models) of such approaches remains questionable. Ideally, it is desirable to model the entire monolithic catalytic reactor using a multi-scale CFD approach in which all pertinent length scales are resolved by the mesh, and no assumptions are made with regard to the relative magnitude of the length and times scales of the underlying physical and chemical processes. Direct numerical simulations are also necessary to benchmark approximate models of the afore-mentioned type. From a numerical standpoint, however, large-scale direct numerical simulation of a catalytic reactor poses several challenges. Estimates by Mazumder (2007) show that modeling a reactor with approximately 500 channels will require about 5 million grid points and more than 200 h of CPU time on a single processor even if a simple one-step reaction mechanism is used to describe the chemistry. If more complex reaction mechanisms comprised of tens of species and tens of reaction steps are considered, the simulation time spans across several days on a single CPU. The memory requirements also go well beyond the limits of a single processor. Parallel computing resources and performance optimizations must be employed to make the calculations feasible with reasonable turnaround time.

Traditionally, CFD codes employ domain decomposition and message passing using MPI to achieve parallelism between the subdomains. While this strategy is necessary for large-scale calculations, it does not utilize the full computational power available within each compute node of modern-day computers. The multi-cores within a CPU and the heterogeneous accelerators (graphical processing units, etc.) in emerging architectures can deliver significant improvement in floating point performance within the processing of each subdomain (Dongarra & Van der Steen, 2012). However, the existing compilers and runtime systems are unable to fully utilize these resources through fully automated code optimizations. As a result, there is a huge gap between the peak floating point performance of a state-of-art machine and the performance achieved by the CFD codes that can be used for simulation of catalytic combustion with detailed chemistry. In fact, many CFD applications reach only 10% or less of a machines peak performance (Hartono, Norris, & Sadayappan, 2009). Manually improving the floating point performance of the CFD applications using optimization techniques targeting emerging architectures are necessary to reduce this gap. These optimizations require detailed profiling of the core serial code, and developing strategies for mapping the performance bottlenecks to the available compute resources. Such efforts for CFD calculations of reacting flow are gaining ground rapidly (Niemeyer & Sung, 2013). Spafford et al. (2010) presented their experiences from accelerating a turbulent reacting flow solver. Shi, Green, Wong, and Oluwole (2011) mapped the matrix inversion step in the backward Euler method to GPUs for solving stiff ODEs in a single kinetic system. Le, Camber, and Cole (2013) demonstrated GPU parallelization of an elementary first order chemistry solver by storing only two rows of the Jacobian in GPU shared memory to overcome the shared memory limit. Though an order of magnitude improvement has been demonstrated in the above methods, the surface chemistry in the aforementioned solvers is simplistic. To the best of our knowledge, performance optimizations targeting emerging hardware resources for simulation of full-scale catalytic reactors with detailed chemistry have not been made in the past.

The objective of the present study is to explore the ramifications of using multi-cores and heterogeneous accelerators (GPUs) for acceleration of CFD calculations of a multi-channel catalytic monolith reactor that includes flow, heat transfer, mass transfer and complex chemical reactions represented by tens of species and reactions. Direct numerical simulation of a full-scale catalytic monolith reactor will require domain decomposition in addition. However, this study is aimed at demonstrating that within each compute node of a domain decomposition based simulation on a multi-node cluster, significant acceleration can be achieved if the hardware that are ubiquitous in most modern-day computers, namely multi-cores and GPUs, are effectively utilized.

This paper provides an initial foundation for full-scale simulation of catalytic reactors with detailed chemistry on emerging compute architectures. The computation of surface chemistry and the material properties has been accelerated by instrumenting the code with multi-threading. The linear solvers and pre-conditioners have been replaced with the GPU counterparts. The species equations are also solved in parallel using the multi-cores. Using these optimizations, drastic improvements (∼4.5×) in overall simulation time has been demonstrated for realistic problems. These optimizations can be used in conjunction with domain decomposition based parallelization.

Section snippets

Governing equations and solution

The modeling framework employed in this research is direct numerical simulation in which all relevant length scales are resolved by the mesh. In such a framework, no sub-grid scale models are necessary.

Sample calculations and results

Two test cases – one in 2D and one in 3D – representative of typical catalytic reactors were selected to investigate the efficacy of the methods just described. There are summarized as follows:

Test Case 1. In this test case, catalytic combustion of pre-mixed methane and air on platinum in a multi-channel catalytic reactor (Fig. 3) is considered. The reactor is assumed to cool externally by convective heat transfer. Two-dimensional (2D) steady-state simulations are conducted. The channels are 2 

Summary and conclusions

Direct numerical simulation of full-scale catalytic monolith reactors with detailed chemistry requires enormous computational resources and time. This is because of the multi-scale nature of the problem in which both the channel scale (∼1 mm) and the reactor scale (∼100 mm) have to be resolved with the mesh. Simplifications can be made to such a multi-scale problem by making assumptions with regard to the uniformity and directionality of temperature, species mass fractions and/or flow

Acknowledgment

This support of the U.S. Department of Energy Office of Science through an STTR Phase I grant (Contract No. DE-SC-0007580, Program Officer Ms. Ceren Susut-Bennett) is gratefully acknowledged.

References (46)

  • S. Mazumder et al.

    Subgrid scale modeling of heterogeneous chemical reactions and transport in full-scale catalytic converters

    Combustion and Flame

    (2002)
  • D. Papadias et al.

    Modeling of high-temperature catalytic combustors: Comparison between theory and experimental data

    Catalysis Today

    (1999)
  • S.V. Patankar et al.

    A calculation procedure for heat, mass, and momentum transfer in three-dimensional parabolic flows

    International Journal of Heat and Mass Transfer

    (1972)
  • L.L. Raja et al.

    A critical evaluation of Navier–Stokes, boundary-layer, and plug-flow models for the simulation of flow and chemistry in a catalytic combustion honeycomb channel

    Catalysis Today

    (2000)
  • Y. Shi et al.

    Redesigning combustion modeling algorithms for graphical processing unit (GPU): Chemical kinetic rate evaluation and ordinary differential equations integration

    Combustion and Flame

    (2011)
  • S. Shuai et al.

    Unsteady temperature fields of monoliths in catalytic converters

    Chemical Engineering Journal

    (2004)
  • J. Stepanek et al.

    Catalyst simulations based on coupling of 3D CFD tools with effective 1D channel models

    Catalysis Today

    (2012)
  • S. Tischer et al.

    Transient three-dimensional simulations of a catalytic combustion monolith using detailed models for heterogeneous and homogeneous reactions and transport phenomena

    Catalysis Today

    (2001)
  • S. Tischer et al.

    Recent advances in numerical modeling of catalytic monolith reactors

    Catalysis Today

    (2005)
  • L. Adhianto et al.

    HPCToolkit: Tools for performance analysis of optimized parallel programs

    Concurrency and Computation: Practice and Experience

    (2010)
  • R.B. Bird et al.

    Transport phenomena

    (2001)
  • Braun, J., Hauber, T., Tobben, H., Windmann, J., Zacke, P., Chatterjee, D., et al. (2002). Three-dimensional simulation...
  • P. Canu et al.

    CFD simulation of reactive flows: Catalytic combustion in a monolith

    AIChE Journal

    (2002)
  • Cited by (0)

    View full text