Model predictive control and moving horizon estimation for adaptive optimal bolus feeding in high-throughput cultivation of E. coli

https://doi.org/10.1016/j.compchemeng.2023.108158Get rights and content

Highlights

  • An MPC and MHE framework is developed for E. coli cultivations in 24 mini-bioreactors.

  • Bolus feeding for mini-scale bioreactors is modeled as an impulsive control system.

  • A complex macro-kinetic growth model is fitted with multi-rate measurements.

  • Three criteria are proposed to classify 24 parallel mini-bioreactors into optimization groups.

Abstract

We discuss the application of a nonlinear model predictive control (MPC) and moving horizon estimation (MHE) framework to achieve optimal operation of E. coli fed-batch cultivations with intermittent bolus feeding. 24 parallel experiments were considered in a high-throughput mini-bioreactor platform at a 10 mL scale. The robotic facility can run up to 48 fed-batch processes in parallel with automated liquid handling, online and at-line analytics. Three main challenges emerge in implementing the model-based monitoring and control framework: First, the inputs are given in an instantaneous pulsed form by bolus injections; second, online and at-line measurement frequencies are severely imbalanced; and third, optimization for the distinctive multiple reactors can be either parallelized or integrated. We address these challenges by incorporating the concept of impulsive control systems, formulating multi-rate MHE with identifiability analysis, and suggesting criteria for deciding the reactor configuration. In this study, we present the key elements and background theory of the implementation with in silico simulations for bacterial fed-batch cultivations.

Introduction

Mathematical descriptions of the dynamics of the microbial systems used in biomanufacturing are key to exploiting bioprocess automation (Leavell et al., 2020). From the early screening phases to the industrial-scale reactor, a consistent model-based approach helps to accelerate product development (Neubauer et al., 2013). The mathematical description of mechanistic models allows for performing optimizations, which leads to cost-effective, consistent, and highly confident bioprocess design. In this context, there has been significant progress related to the model-based bioprocess automation, such as structuring a mechanistic model that describes key metabolic pathways, parameter estimation and design of experiment, optimal control, and modularized computational frameworks (Gomes et al., 2015, Narayanan et al., 2020, Hemmerich et al., 2021, Herwig et al., 2021).

One of the distinguishing characteristics of biological systems is that the growth of the microorganism exhibits a high level of uncertainty. Moreover, the most used cultivation strategy is fed-batch, in which no steady-state exists and the process variables vary over a wide range. Hence, a large amount of data is needed to obtain a mathematical model which is capable of describing the non-steady-state dynamics of highly nonlinear and complex systems such as cells (Cruz Bournazou et al., 2017). The recent development of high-throughput (HT) technology allows for performing a large number of laborious and time-consuming experiments by automatizing, parallelizing, and miniaturizing the experimental facilities (Puskeiler et al., 2005, Bunzel et al., 2018, Hemmerich et al., 2018). Liquid handling stations support the parallel cultivation of a large number of mini-bioreactors, based on the automated sampling, online and at-line analytics operation, real-time control, and data acquisition (Kusterer et al., 2008, Tai et al., 2015, Haby et al., 2019, Hans et al., 2020b). HT technology plays an important role, especially in the early stage of bioprocess development. Very large amounts of data from testing the cell clones under different conditions such as media, pH, temperature, induction strengths, and feeding strategies become available (Hans et al., 2020a) allowing us to understand the process better and to find the best-operating conditions in very large search spaces. This has naturally led to the enhancement of the conditional screening and strain phenotyping (Schmideder et al., 2016, Sawatzki et al., 2018, Janzen et al., 2019, Fink et al., 2021).

To exploit the full potential of the HT bioprocess development (HTBD), it is essential to couple the model-based methods with robotic facilities. The reliability of computed decisions depends on the accuracy of model predictions, e.g., each cell type and organism needs tailored feeding strategies for optimal growth or product formation. Because of limited reproducibility at the small-scale reactors at the μ L and mL scale, these strategies need to be adaptively controlled. In addition, monitoring of cultivations is often challenging because of the limited accuracy of online sensors, e.g., small delays and shifts in oxygen signals, or large delays in the processing of liquid samples. Finally, the satisfaction of experimental constraints is critical, such as minimum levels of sugar and/or dissolved oxygen concentrations to minimize specific stresses, i.e., mainly glucose excess, limitation and starvation, and oxygen exhaustion (Delvigne et al., 2009).

In previous works, intermittent-feeding strategies have been derived from established exponential feeding strategies considering the (previously or online identified) maximum specific growth rate of the organism (Sawatzki et al., 2018, Anane et al., 2019b, Hans et al., 2020a). Because of the limited information and feedback, it has been observed that this approach can lack robustness in real applications. There are only a few studies that incorporate model-based methods for HT experiments. However, the studies consider optimal experimental design for the model fitting instead of optimal operation for maximizing biomass (Cruz Bournazou et al., 2017, Barz et al., 2018, Kim et al., 2021a).

Model predictive control (MPC) and moving horizon estimation (MHE) are popular methods in engineering (Rawlings et al., 2017) and there has been a large number of applications for fed-batch bioreactor cultivations. Given the state trajectory from the pre-defined operating strategy, MPC has been applied for the optimal tracking control of the bioprocesses described with basic Monod equations (Ramaswamy et al., 2005, Tebbani et al., 2008), and further extended to economic objectives such as to maximize the product (Ashoori et al., 2009, Raftery et al., 2017). In the presence of measurement and model uncertainties, optimal state or parameter estimators such as Kalman filter (Markana et al., 2018) or MHE (Abdollahi and Dubljevic, 2012, del Rio-Chanona et al., 2016) are combined with MPC in order to adapt to the data. Various optimization methods have been studied such as maximization principle (Pčolka and Čelikovskỳ, 2016, Luna and Martínez, 2017) and evolutionary strategies (Freitas et al., 2017). As an alternative to MPC, the reinforcement learning (RL) method has been studied to reduce the online computation time by obtaining the closed-loop policy and for optimal operation even without the mechanistic model (Martínez et al., 2013, Petsagkourakis et al., 2020, Kim et al., 2021b).

Although several studies have proven that the model-based approach comprising a controller and estimator is promising for automated bioprocess operation (Lucia et al., 2017a), it has, to our knowledge, never been implemented in parallel mini-bioreactors for HTBD. The extension to this application is not trivial and has important challenges that can lead to poorly operated experiments. We discuss the most relevant challenges and how they have been tackled in the presented framework. First, inputs are given in a pulse form by bolus injections. In highly parallelized milliliter scale mini-bioreactors for example, individual feeding with high accuracy is very challenging and costly (Faust et al., 2014). As an alternative, pulse-based feeding has been widely utilized. In contrast to continuous feeding, intermittent feeding fails to achieve high cell density cultivation, because it induces large heterogeneities in the operating conditions such as oscillating pH, oxygen, temperature, glucose concentrations, and toxic compounds (Neubauer et al., 2013). This can, however, when properly designed and operated using scale-down techniques, be used to mimic the industrial scale heterogeneity (Neubauer and Junne, 2016). According to Anane et al. (2019b), the combination of a pulse-based scale-down approach is successful to test strain robustness and physiological constraints at the early stages of bioprocess development. The pulse-based (bolus) feeding is different compared to the standard process control problems where systems have continuous input and dynamics. This requires a different approach, namely, impulsive control systems (Yang, 2001).

Second, the measurements are multi-rate, the sampling times between the parallel bioreactors are not aligned, and the sample delays are on the order of tens of minutes such that a multi-rate formulation of the MHE is necessary. The system is limited to extracellular measurements, which makes it very difficult to generate sufficient information to properly identify mechanistic models that aim to describe a highly nonlinear and large biochemical network. For this reason, efficient methods to assure a well-posed state and parameter estimation problem are needed.

Third, given the configuration of multiple cultivation conditions and bioreactors, the proper design of the experimental campaign and distribution of the operating strategies are not trivial. The information in parallel systems (e.g. replicates) needs to be efficiently exploited to improve the probability of finding the optimal process condition.

Our contribution focuses on the computational model-based framework, namely, MPC and MHE for the optimization of the cultivation conditions for maximal growth of E. coli in parallel mini-bioreactors. Three bottlenecks encountered in the HT experiment, bolus injection, multi-rate measurements, and multiple bioreactors, are addressed: First, we illustrate how to design the objective and constraint function for the impulsive control systems. Unlike the zero-order hold discretization which has been used for impulsive systems, a full-discretization method is implemented to capture the fast dynamics happening within the pulse-feed interval. Second, we propose an arrival cost design and identifiable parameter selection method for the multi-rate and partially ill-conditioned MHE. Third, for the purpose of determining the reactor configuration for the MHE optimization, three criteria, size of the identifiable parameter subset, root mean squared error of the MHE, and computational time are suggested. The developed framework is validated through in silico studies on fed-batch cultivations of E. coli. As a proof-of-concept, we design a 24 parallel cultivation experiment, which is divided into 8 different conditions with 3 replicates. We focus on whether the MPC and MHE are capable of providing a consistent/feasible pulse-feeding strategy regardless of the operating condition. The experimental validation is conducted in the companion paper (Krausch et al., 2022).

The remainder of this paper is organized as follows: Section 2 describes the in silico experiments of mini-bioreactors and the challenges of the HT experiment. Section 3 introduces model-based methods, MHE and MPC. In Section 4, the simulation results are discussed. Finally, a discussion and some concluding remarks are provided in Section 5.

Section snippets

Experimental setup, sensors, and automated liquid handling

The robotic facility of the HTBD platform is capable of performing 24 parallel cultivations. The liquid handling station assists in sampling, measuring, glucose feeding, medium balancing, and pH control (acid and base). The 24 mini-bioreactors are placed in three columns and eight rows, and we put a numeric order to the columns (i.e., 1, 2, and 3), and an alphabetical order to the rows (i.e., A, B, …, H). Based on this configuration, reactors that have the same columns are replicates and the HT

Problem description and formulation

We describe the HT experiment and the macro-kinetic growth model in formal expressions. The states x, manipulated variables u, and measured variables y comprise of x=X,S,A,DOTm,P,Vy=X,S,A,DOTm,Pu=Δv

The experimental procedures are characterized by the following finite sets : R=(row,col)|rowA,B,H,col1,2,3U=10k (min)|k0,1,2,Mr,y=40+20(i1)+60k (min)|k0,1,2,,r=(i,)R,yX,S,A,P30k(s)|k0,1,2,,rR,y=DOTmwhere R is the index set of the mini-bioreactors; U is the discrete pulse-feeding times; M

Determination of the MHE configuration and identifiability analysis

The in silico cultivation for 24 mini-bioreactors is performed in parallel, with the proposed model-based methods. To generate the in silico data, we add random uniform noise with 20% of their scale to the growth parameters in Table 2 for each row and add additional uniform noise with 5% of their scale to the replicate in each column. For the reactor-dependent parameters in Table 3, 20% uniform noises are added for all bioreactors. Moreover, Gaussian noise with 5% variance to the scales is

Concluding remarks

In this study, we develop a macro-kinetic growth model-based MHE and MPC framework for high-throughput experiments with 24 parallel mini-bioreactors. Three crucial aspects of the system are addressed. First, the pulse-based feed is formulated as an impulsive control system. We perform the analysis of the objective and constraint function and implemented a full-discretization method for solving the MPC. Second, multi-rate MHE is formulated. The MHE method utilizes parameter sensitivity as an

CRediT authorship contribution statement

Jong Woo Kim: Conceptualization, Methodology, Software, Writing – original draft. Niels Krausch: Conceptualization, Methodology, Software, Writing – original draft. Judit Aizpuru: Methodology, Software. Tilman Barz: Methodology, Formal analysis, Writing – review & editing. Sergio Lucia: Methodology, Software, Formal analysis, Writing – review & editing. Peter Neubauer: Supervision, Writing – review & editing. Mariano Nicolas Cruz Bournazou: Supervision, Writing – review & editing.

Declaration of Competing Interest

The authors declare that they have no known competing financial interests or personal relationships that could have appeared to influence the work reported in this paper.

Acknowledgments

This work was supported by the German Federal Ministry of Education and Research through the Program International Future Labs for Artificial Intelligence (KIWI Biolab - Grant number 01DD20002A).

References (70)

  • ElsheikhM. et al.

    A comparative review of multi-rate moving horizon estimation schemes for bioprocess applications

    Comput. Chem. Eng.

    (2021)
  • HabyB. et al.

    Integrated robotic mini bioreactor platform for automated, parallel microbial cultivation with online data handling and process control

    SLAS Technol.: Transl. Life Sci. Innov.

    (2019)
  • JounedM.A. et al.

    Event driven modelling for the accurate identification of metabolic switches in fed-batch culture of S. cerevisiae

    Biochem. Eng. J.

    (2022)
  • KimJ.W. et al.

    Model-based reinforcement learning and predictive control for two-stage optimal control of fed-batch bioreactor

    Comput. Chem. Eng.

    (2021)
  • KrämerS. et al.

    Multirate state estimation using moving horizon estimation

    IFAC Proc. Vol.

    (2005)
  • LeavellM.D. et al.

    High-throughput screening for improved microbial cell factories, perspective and promise

    Curr. Opin. Biotechnol.

    (2020)
  • López-NegreteR. et al.

    A moving horizon estimator for processes with multi-rate measurements: A nonlinear programming sensitivity approach

    J. Process Control

    (2012)
  • LuciaS. et al.

    Adaptive nonlinear predictive control and estimation of microaerobic processes

    IFAC-PapersOnLine

    (2017)
  • LuciaS. et al.

    Rapid development of modular and sustainable nonlinear model predictive control solutions

    Control Eng. Pract.

    (2017)
  • LunaM.F. et al.

    Iterative modeling and optimization of biomass production using experimental feedback

    Comput. Chem. Eng.

    (2017)
  • MarkanaA. et al.

    Multi-criterion control of a bioprocess in fed-batch reactor using EKF based economic model predictive control

    Chem. Eng. Res. Des.

    (2018)
  • MartínezE.C. et al.

    Dynamic optimization of bioreactors using probabilistic tendency models and Bayesia active learning

    Comput. Chem. Eng.

    (2013)
  • PetsagkourakisP. et al.

    Reinforcement learning for batch bioprocess optimization

    Comput. Chem. Eng.

    (2020)
  • QuC.C. et al.

    Computation of arrival cost for moving horizon estimation via unscented Kalman filtering

    J. Process Control

    (2009)
  • RamaswamyS. et al.

    Control of a continuous bioreactor using model predictive control

    Process Biochem.

    (2005)
  • ShenL. et al.

    Bilevel parameters identification for the multi-stage nonlinear impulsive system in microorganisms fed-batch cultures

    Nonlinear Anal. RWA

    (2008)
  • TebbaniS. et al.

    Open-loop optimization and trajectory tracking of a fed-batch bioreactor

    Chem. Eng. Process.: Process Intensif.

    (2008)
  • YangX. et al.

    Recent progress in impulsive control systems

    Math. Comput. Simulation

    (2019)
  • YinX. et al.

    Distributed moving horizon state estimation of two-time-scale nonlinear systems

    Automatica

    (2017)
  • AlexanderR. et al.

    Challenges and opportunities on nonlinear state estimation of chemical and biochemical processes

    Processes

    (2020)
  • AnaneE. et al.

    Modelling concentration gradients in fed-batch cultivations of E. coli - towards the flexible design of scale-down experiments

    J. Chem. Technol. Biotechnol.

    (2019)
  • AnderssonJ.A.E. et al.

    CasADi – A software framework for nonlinear optimization and optimal control

    Math. Program. Comput.

    (2019)
  • BieglerL.T.

    Nonlinear Programming: Concepts, Algorithms, and Applications to Chemical Processes

    (2010)
  • Cruz BournazouM.N. et al.

    Online optimal experimental re-design in robotic parallel fed-batch cultivation facilities

    Biotechnol. Bioeng.

    (2017)
  • DelvigneF. et al.

    Bioreactor mixing efficiency modulates the activity of a prpos::gfp reporter gene in e. coli

    Microb. Cell Factories

    (2009)
  • Cited by (7)

    View all citing articles on Scopus
    View full text