Log In Sign Up

Reinforcement Learning under Model Risk for Biomanufacturing Fermentation Control

by   Bo Wang, et al.

In the biopharmaceutical manufacturing, fermentation process plays a critical role impacting on productivity and profit. Since biotherapeutics are manufactured in living cells whose biological mechanisms are complex and have highly variable outputs, in this paper, we introduce a model-based reinforcement learning framework accounting for model risk to support bioprocess online learning and guide the optimal and robust customized stopping policy for fermentation process. Specifically, built on the dynamic mechanisms of protein and impurity generation, we first construct a probabilistic model characterizing the impact of underlying bioprocess stochastic uncertainty on impurity and protein growth rates. Since biopharmaceutical manufacturing often has very limited data during the development and early stage of production, we derive the posterior distribution quantifying the process model risk, and further develop the Bayesian rule based knowledge update to support the online learning on underlying stochastic process. With the prediction risk accounting for both bioprocess stochastic uncertainty and model risk, the proposed reinforcement learning framework can proactively hedge all sources of uncertainties and support the optimal and robust customized decision making. We conduct the structural analysis of optimal policy and study the impact of model risk on the policy selection. We can show that it asymptotically converges to the optimal policy obtained under perfect information of underlying stochastic process. Our case studies demonstrate that the proposed framework can greatly improve the biomanufacturing industrial practice.


page 18

page 19

page 20


Green Simulation Assisted Reinforcement Learning with Model Risk for Biomanufacturing Learning and Control

Biopharmaceutical manufacturing faces critical challenges, including com...

Policy Optimization in Bayesian Network Hybrid Models of Biomanufacturing Processes

Biopharmaceutical manufacturing is a rapidly growing industry with impac...

Reinforcement Learning-based Defect Mitigation for Quality Assurance of Additive Manufacturing

Additive Manufacturing (AM) is a powerful technology that produces compl...

Green Simulation Assisted Policy Gradient to Accelerate Stochastic Process Control

This study is motivated by the critical challenges in the biopharmaceuti...

Doubly Robust Interval Estimation for Optimal Policy Evaluation in Online Learning

Evaluating the performance of an ongoing policy plays a vital role in ma...

A model-based framework for integrated process design and flexibility analysis

Process development is typically associated with lengthy wet-lab experim...