This is not the latest version of this item. The latest version can be found here.
Controlling Level of Unconsciousness by Titrating Propofol with Deep Reinforcement Learning
Name
2008.12333.pdf
Description
Accepted version
Size
2.74 MB
Format
Adobe PDF
Checksum (MD5)
ae47da25e34f68af5c689aaba8f54284
Author(s)
Schamberg, Gabriel
Badgeley, Marcus
Brown, Emery N
Date Issued
2020
Journal
Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics)
Publisher
Springer International Publishing
Citation
Schamberg, Gabriel, Badgeley, Marcus and Brown, Emery N. 2020. "Controlling Level of Unconsciousness by Titrating Propofol with Deep Reinforcement Learning." Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics), 12299.
Version
Author's final manuscript
Abstract
Reinforcement Learning (RL) can be used to fit a mapping from patient state to a medication regimen. Prior studies have used deterministic and value-based tabular learning to learn a propofol dose from an observed anesthetic state. Deep RL replaces the table with a deep neural network and has been used to learn medication regimens from registry databases. Here we perform the first application of deep RL to closed-loop control of anesthetic dosing in a simulated environment. We use the cross-entropy method to train a deep neural network to map an observed anesthetic state to a probability of infusing a fixed propofol dosage. During testing, we implement a deterministic policy that transforms the probability of infusion to a continuous infusion rate. The model is trained and tested on simulated pharmacokinetic/pharmacodynamic models with randomized parameters to ensure robustness to patient variability. The deep RL agent significantly outperformed a proportional-integral-derivative controller (median absolute performance error 1.7% ± 0.6 and 3.4% ± 1.2). Modeling continuous input variables instead of a table affords more robust pattern recognition and utilizes our prior domain knowledge. Deep RL learned a smooth policy with a natural interpretation to data scientists and anesthesia care providers alike.
Terms of Use
Creative Commons Attribution-Noncommercial-Share Alike
Persistent DSpace Link
DOI of Published Version
10.1007/978-3-030-59137-3_3
