research-article

Optimizing Substance Use Treatment Selection Using Reinforcement Learning

Authors:
Matt Baucum

Florida State University, Tallahassee, FL, USA

Florida State University, Tallahassee, FL, USA

0000-0002-2699-167X
View Profile

,
Anahita Khojandi

University of Tennessee, Knoxville, TN, USA

University of Tennessee, Knoxville, TN, USA

0000-0001-6818-2048
View Profile

,
Carole Myers

University of Tennessee, Knoxville, TN, USA

University of Tennessee, Knoxville, TN, USA

0000-0001-5766-9651
View Profile

,
Larry Kessler

University of Tennessee, Knoxville, TN, USA

University of Tennessee, Knoxville, TN, USA

0000-0002-2953-8838
View Profile

ACM Transactions on Management Information Systems Volume 14 Issue 2Article No.: 13pp 1–30https://doi.org/10.1145/3563778

Published:25 March 2023Publication History

ACM Transactions on Management Information Systems

Abstract

Substance use disorder (SUD) exacts a substantial economic and social cost in the United States, and it is crucial for SUD treatment providers to match patients with feasible, effective, and affordable treatment plans. The availability of large SUD patient datasets allows for machine learning techniques to predict patient-level SUD outcomes, yet there has been almost no research on whether machine learning can be used to optimize or personalize which treatment plans SUD patients receive. We use contextual bandits (a reinforcement learning technique) to optimally map patients to SUD treatment plans, based on dozens of patient-level and geographic covariates. We also use near-optimal policies to incorporate treatments’ time-intensiveness and cost into our recommendations, to aid treatment providers and policymakers in allocating treatment resources. Our personalized treatment recommendation policies are estimated to yield higher remission rates than observed in our original dataset, and they suggest clinical insights to inform future research on data-driven SUD treatment matching.

REFERENCES

[1] Centers for Disease Control (CDC) Agency for Toxic Substances and Disease Registry. [n. d.]. CDC/ATSDR SVI Data and Documentation Download. https://www.atsdr.cdc.gov/placeandhealth/svi/data_documentation_download.html.Google Scholar
[2] National Institute on Drug Abuse (NIDA). [n. d.]. Costs of Substance Abuse. Retrieved December 6, 2021, from https://archives.drugabuse.gov/trends-statistics/costs-substance-abuse.Google Scholar
[3] Arete Recovery. [n. d.]. Is a 14-day Rehab Program Right for You? https://areterecovery.com/treatment/14-day-inpatient/.Google Scholar
[4] Substance Abuse and Mental Health Services Administration (SAMHSA). [n. d.]. TEDS-D Information. https://wwwdasis.samhsa.gov/webt/information.htm.Google Scholar
[5] Substance Abuse and Mental Health Services Administration (SAMHSA). 2000. Chapter 6 – Funding and Policy Issues. Retrieved June 8, 2022, from https://www.ncbi.nlm.nih.gov/books/NBK64279/.Google Scholar
[6] Substance Abuse and Mental Health Services Administration (SAMHSA). 2006-2019. Treatment Episode Data Set (TEDS) Discharges. https://www.datafiles.samhsa.gov/dataset/teds-d-2019-ds0001-teds-d-2019-ds0001.Google Scholar
[7] Substance Abuse and Mental Health Services Administration (SAMHSA). 2013. Drug Abuse Warning Network, 2011: National Estimates of Drug-related Emergency Department Visits. Technical report. https://www.samhsa.gov/data/sites/default/files/DAWN2k11ED/DAWN2k11ED/DAWN2k11ED.pdf.Google Scholar
[8] National Institute on Drug Abuse (NIDA). 2018. How Long Does Drug Addiction Treatment Usually Last? https://www.drugabuse.gov/publications/principles-drug-addiction-treatment-research-based-guide-third-edition/frequently-asked-questions/how-long-does-drug-addiction-treatment-usually-last.Google Scholar
[9] National Institute on Drug Abuse (NIDA). 2018. Is Drug Addiction Treatment Worth the Cost? Retrieved December 6, 2021, from https://www.drugabuse.gov/publications/principles-drug-addiction-treatment-research-based-guide-third-edition/frequently-asked-questions/drug-addiction-treatment-worth-its-cost.Google Scholar
[10] White House Archives. 2019. The Full Cost of the Opioid Crisis: $2.5 Trillion over Four Years. Retrieved December 6, 2021, from https://trumpwhitehouse.archives.gov/articles/full-cost-opioid-crisis-2-5-trillion-four-years/.Google Scholar
[11] National Institute on Drug Abuse (NIDA). 2020. Criminal Justice Drug Facts. Retrieved June 8, 2022, from https://nida.nih.gov/download/23025/criminal-justice-drugfacts.pdf?v=25dde14276b2fa252318f2c573407966.Google Scholar
[12] National Institute on Drug Abuse (NIDA). 2020. Treatment and Recovery. https://www.drugabuse.gov/publications/drugs-brains-behavior-science-addiction/treatment-recovery.Google Scholar
[13] National Institute on Drug Abuse (NIDA). 2020. Types of Treatment Programs. https://www.drugabuse.gov/publications/principles-drug-addiction-treatment-research-based-guide-third-edition/drug-addiction-treatment-in-united-states/types-treatment-programs.Google Scholar
[14] National Institute on Drug Abuse (NIDA). 2021. COVID-19 and Substance Use. https://www.drugabuse.gov/drug-topics/comorbidity/covid-19-substance-use.Google Scholar
[15] The Pew Charitable Trusts. 2022. 2022 Drug Arrests Stayed High Even as Imporisonment Fell from 2009 to 2019. Retrieved June 8, 2022, from https://www.pewtrusts.org/-/media/assets/2022/02/drug-arrests-stayed-high-even-as-imprisonment-fell-from-2009-to-2019.pdf.Google Scholar
[16] Substance Abuse and Mental Health Services Administration (SAMHSA). 2022. Preliminary Findings From Drug-Related Emergency Department Visits, 2021. Technical report. https://www.samhsa.gov/data/report/dawn-2021-preliminary-findings-report.Google Scholar
[17] Abramson A.. 2021. Substance Use During the Pandemic. https://www.apa.org/monitor/2021/03/substance-use-pandemic.Google Scholar
[18] Acion Laura, Kelmansky Diana, Laan Mark van der, Sahker Ethan, Jones DeShauna, and Arndt Stephan. 2017. Use of a machine learning framework to predict substance use disorder treatment success. PloS One 12, 4 (2017), e0175383.Google ScholarCross Ref
[19] Agrawal Shipra and Devanur Nikhil. 2016. Linear contextual bandits with knapsacks. Advances in Neural Information Processing Systems 29 (2016).Google Scholar
[20] Dionissi Aliprantis, Kyle Fee, and Mark Schweitzer. 2019. Opioids and the labor market. Working paper, Federal Reserve Bank of Cleveland. https://www.clevelandfed.org/-/media/project/clevelandfedtenant/clevelandfedsite/publications/working-papers/2019/wp-1807r2-opioids-and-the-labor-market-pdf.pdf.Google Scholar
[21] Ammerman Robert T., Kolko David J., Kirisci Levent, Blackson Timothy C., and Dawes Michael A.. 1999. Child abuse potential in parents with histories of substance use disorder. Child Abuse & Neglect 23, 12 (1999), 1225–1238.Google ScholarCross Ref
[22] Andersson Helle Wessel, Wenaas Merethe, and Nordfjærn Trond. 2019. Relapse after inpatient substance use treatment: A prospective cohort study among users of illicit substances. Addictive Behaviors 90 (2019), 222–228.Google ScholarCross Ref
[23] Ayer Turgay, Alagoz Oguzhan, and Stout Natasha K. 2012. OR Forum–A POMDP approach to personalize mammography screening decisions. Operations Research 60, 5 (2012), 1019–1034.Google ScholarDigital Library
[24] Aziz Maryam, Kaufmann Emilie, and Riviere Marie-Karelle. 2021. On multi-armed bandit designs for dose-finding clinical trials. Journal of Machine Learning Research 22, 14 (2021), 1–38.Google Scholar
[25] Barenholtz Elan, Fitzgerald Nicole D., and Hahn William Edward. 2020. Machine-learning approaches to substance-abuse research: Emerging trends and their implications. Current Opinion in Psychiatry 33, 4 (2020), 334–342.Google ScholarCross Ref
[26] Bastani Hamsa and Bayati Mohsen. 2020. Online decision making with high-dimensional covariates. Operations Research 68, 1 (2020), 276–294.Google ScholarDigital Library
[27] Baucum M., Khojandi A., and Vasudevan R.. 2020. Improving deep reinforcement learning with transitional variational autoencoders: A healthcare application. IEEE Journal of Biomedical and Health Informatics 25, 6 (2020), 2273–2280. Google ScholarCross Ref
[28] Berger Lawrence M., Cancian Maria, Cuesta Laura, and Noyes Jennifer L.. 2016. Families at the intersection of the criminal justice and child protective services systems. Annals of the American Academy of Political and Social Science 665, 1 (2016), 171–194.Google ScholarCross Ref
[29] Bhalla Ish P., Stefanovics Elina A., and Rosenheck Robert A.. 2017. Clinical epidemiology of single versus multiple substance use disorders: Polysubstance use disorder. Medical Care 55 (2017), S24–S32.Google ScholarCross Ref
[30] Biswas Arpita, Aggarwal Gaurav, Varakantham Pradeep, and Tambe Milind. 2021. Learning index policies for restless bandits with application to maternal healthcare. In Proceedings of the 20th International Conference on Autonomous Agents and MultiAgent Systems. 1467–1468.Google ScholarDigital Library
[31] Bothe Melanie K., Dickens Luke, Reichel Katrin, Tellmann Arn, Ellger Björn, Westphal Martin, and Faisal Ahmed A.. 2013. The use of reinforcement learning algorithms to meet the challenges of an artificial pancreas. Expert Review of Medical Devices 10, 5 (2013), 661–673.Google ScholarCross Ref
[32] Bouneffouf Djallel and Rish Irina. 2019. A survey on practical applications of multi-armed and contextual bandits. arXiv preprint arXiv:1904.10040 (2019).Google Scholar
[33] Brecht Mary-Lynn and Herbeck Diane. 2014. Time to relapse following treatment for methamphetamine use: A long-term perspective on patterns and predictors. Drug and Alcohol Dependence 139 (2014), 18–25.Google ScholarCross Ref
[34] Breiman Leo. 2001. Random forests. Machine Learning 45, 1 (2001), 5–32.Google ScholarDigital Library
[35] Brightwell Graham, Kenyon Claire, and Paugam-Moisy Hélène. 1996. Multilayer neural networks: One or two hidden layers?Advances in Neural Information Processing Systems 9 (1996), 148–154.Google Scholar
[36] Calabria Bianca, Degenhardt Louisa, Briegleb Christina, Vos Theo, Hall Wayne, Lynskey Michael, Callaghan Bridget, Rana Umer, and McLaren Jennifer. 2010. Systematic review of prospective studies investigating “remission” from amphetamine, cannabis, cocaine or opioid dependence. Addictive Behaviors 35, 8 (2010), 741–749.Google ScholarCross Ref
[37] Chohlas-Wood Alex, Coots Madison, Brunskill Emma, and Goel Sharad. 2021. Learning to be fair: A consequentialist approach to equitable decision-making. arXiv preprint arXiv:2109.08792 (2021).Google Scholar
[38] Cicchetti Dante and Handley Elizabeth D.. 2019. Child maltreatment and the development of substance use and disorder. Neurobiology of Stress 10 (2019), 100144.Google ScholarCross Ref
[39] Copenhaver Michael M., Bruce R. Douglas, and Altice Frederick L.. 2007. Behavioral counseling content for optimizing the use of buprenorphine for treatment of opioid dependence in community-based settings: A review of the empirical evidence. American Journal of Drug and Alcohol Abuse 33, 5 (2007), 643–654.Google ScholarCross Ref
[40] Cowell Alexander, McCarty Dennis, and Woodward Albert. 2003. Impact of federal substance abuse block grants on state substance abuse spending: Literature and data review. Journal of Mental Health Policy and Economics 6, 4 (2003), 173–180.Google Scholar
[41] Crummy Elizabeth A., O’Neal Timothy J., Baskin Britahny M., and Ferguson Susan M.. 2020. One is not enough: Understanding and modeling polysubstance use. Frontiers in Neuroscience 14 (2020), 569.Google ScholarCross Ref
[42] Currie Janet and Widom Cathy Spatz. 2010. Long-term consequences of child abuse and neglect on adult economic well-being. Child Maltreatment 15, 2 (2010), 111–120.Google ScholarCross Ref
[43] Currie Janet and Tekin Erdal. 2012. Understanding the cycle childhood maltreatment and future crime. Journal of Human Resources 47, 2 (2012), 509–549.Google ScholarCross Ref
[44] Daughters Stacey B., Lejuez C. W., Bornovalova Marina A., Kahler Christopher W., Strong David R., and Brown Richard A.. 2005. Distress tolerance as a predictor of early treatment dropout in a residential substance abuse treatment facility.Journal of Abnormal Psychology 114, 4 (2005), 729.Google ScholarCross Ref
[45] Davis Margaret I. and Jason Leonard A.. 2005. Sex differences in social support and self-efficacy within a recovery community. American Journal of Community Psychology 36, 3–4 (2005), 259–274.Google ScholarCross Ref
[46] Decker Kathleen P., Peglow Stephanie L., Samples Carl R., and Cunningham Tina D.. 2017. Long-term outcomes after residential substance use treatment: Relapse, morbidity, and mortality. Military Medicine 182, 1–2 (2017), e1589–e1595.Google Scholar
[47] Degenhardt Louisa, Bucello Chiara, Mathers Bradley, Briegleb Christina, Ali Hammad, Hickman Matt, and McLaren Jennifer. 2011. Mortality among regular or dependent users of heroin and other opioids: A systematic review and meta-analysis of cohort studies. Addiction 106, 1 (2011), 32–51.Google ScholarCross Ref
[48] Desai Niraj M., Mange Kevin C., Crawford Michael D., Abt Peter L., Frank Adam M., Markmann Joseph W., Velidedeoglu Ergun, Chapman William C., and Markmann James F.. 2004. Predicting outcome after liver transplantation: Utility of the model for end-stage liver disease and a newly derived discrimination function1. Transplantation 77, 1 (2004), 99–106.Google ScholarCross Ref
[49] Dobkin Patricia L., Civita Mirella De, Paraherakis Antonios, and Gill Kathryn. 2002. The role of functional social support in treatment retention and outcomes among outpatient adult substance abusers. Addiction 97, 3 (2002), 347–356.Google ScholarCross Ref
[50] Dudík Miroslav, Erhan Dumitru, Langford John, and Li Lihong. 2014. Doubly robust policy evaluation and optimization. Statist. Sci. 29, 4 (2014), 485–511.Google ScholarCross Ref
[51] Dudík Miroslav, Langford John, and Li Lihong. 2011. Doubly robust policy evaluation and learning. arXiv preprint arXiv:1103.4601 (2011).Google Scholar
[52] Durand Louise, Boland Fiona, O’Driscoll Denis, Bennett Kathleen, Barry Joseph, Keenan Eamon, Fahey Tom, and Cousins Gráinne. 2021. Factors associated with early and later dropout from methadone maintenance treatment in specialist addiction clinics: A six-year cohort study using proportional hazards frailty models for recurrent treatment episodes. Drug and Alcohol Dependence 219 (2021), 108466.Google ScholarCross Ref
[53] Eckenrode John, Laird Molly, and Doris John. 1993. School performance and disciplinary problems among abused and neglected children.Developmental Psychology 29, 1 (1993), 53.Google ScholarCross Ref
[54] Farajtabar Mehrdad, Chow Yinlam, and Ghavamzadeh Mohammad. 2018. More robust doubly robust off-policy evaluation. In International Conference on Machine Learning. PMLR, 1447–1456.Google Scholar
[55] Fletcher Jesse B. and Reback Cathy J.. 2021. Optimizing outpatient treatment outcomes among methamphetamine-using gay and bisexual men through a computerized depression intervention. Journal of Substance Abuse Treatment 136 (2021), 108663.Google Scholar
[56] French Michael T., Popovici Ioana, and Tapsell Lauren. 2008. The economic costs of substance abuse treatment: Updated estimates and cost bands for program assessment and reimbursement. Journal of Substance Abuse Treatment 35, 4 (2008), 462–469.Google ScholarCross Ref
[57] Ghosh Abhishek, Sharma Nidhi, Subodh B. N., Basu Debasish, Mattoo Surendra Kumar, and Pillai Renjith R.. 2022. Predictors of dropout from an outpatient treatment program for substance use disorders in india: A retrospective cohort study of patients registered over a 10-year period (2009–2018). International Journal of Mental Health and Addiction 20 (2022), 943–955.Google ScholarCross Ref
[58] Gottheil Edward, McLellan A. Thomas, and Druley Keith A.. 1992. Length of stay, patient severity and treatment outcome: Sample data from the field of alcoholism.Journal of Studies on Alcohol 53, 1 (1992), 69–75.Google ScholarCross Ref
[59] Han Dae-Hee, Lee Shieun, and Seo Dong-Chul. 2020. Using machine learning to predict opioid misuse among US adolescents. Preventive Medicine 130 (2020), 105886.Google ScholarCross Ref
[60] Harris Matthew C., Kessler Lawrence M., Murray Matthew N., and Glenn Beth. 2020. Prescription opioids and labor market pains the effect of schedule II opioids on labor force participation and unemployment. Journal of Human Resources 55, 4 (2020), 1319–1364.Google ScholarCross Ref
[61] Harvey Ronald, Jason Leonard A., and Ferrari Joseph R.. 2016. Substance abuse relapse in Oxford House recovery homes: A survival analysis evaluation. Substance Abuse 37, 2 (2016), 281–285.Google ScholarCross Ref
[62] Hasan Md Mahmudul, Young Gary J., Shi Jiesheng, Mohite Prathamesh, Young Leonard D., Weiner Scott G., and Md. Noor-E-Alam. 2021. A machine learning based two-stage clinical decision support system for predicting patients’ discontinuation from opioid use disorder treatment: Retrospective observational study. BMC Medical Informatics and Decision Making 21, 1 (2021), 1–21.Google ScholarCross Ref
[63] Hayashida Motoi. 1998. An overview of outpatient and inpatient detoxification. Alcohol Health and Research World 22, 1 (1998), 44.Google Scholar
[64] Hjemsæter Arne Jan, Bramness Jørgen G., Drake Robert, Skeie Ivar, Monsbakken Bent, Benth Jūratė Šaltytė, and Landheim Anne S.. 2019. Mortality, cause of death and risk factors in patients with alcohol use disorder alone or poly-substance use disorders: A 19-year prospective cohort study. BMC Psychiatry 19, 1 (2019), 1–9.Google ScholarCross Ref
[65] Hubbard Robert L., Flynn Patrick M., Craddock S. Gail, and Fletcher Bennett W.. 2001. Relapse after drug abuse treatment. In Relapse and Recovery in Addictions. Yale University Press, 109–121.Google Scholar
[66] Jin Yujia and Sidford Aaron. 2020. Efficiently solving MDPs with stochastic mirror descent. In International Conference on Machine Learning. PMLR, 4890–4900.Google Scholar
[67] Jing Yankang, Hu Ziheng, Fan Peihao, Xue Ying, Wang Lirong, Tarter Ralph E., Kirisci Levent, Wang Junmei, Vanyukov Michael, and Xie Xiang-Qun. 2020. Analysis of substance use and its outcomes by machine learning I. Childhood evaluation of liability to substance use disorder. Drug and Alcohol Dependence 206 (2020), 107605.Google ScholarCross Ref
[68] Kassani Aziz, Niazi Mohsen, Hassanzadeh Jafar, and Menati Rostam. 2015. Survival analysis of drug abuse relapse in addiction treatment centers. International Journal of High Risk Behaviors & Addiction 4, 3 (2015), e23402. DOI:Google ScholarCross Ref
[69] Kinreich Sivan, McCutcheon Vivia V., Aliev Fazil, Meyers Jacquelyn L., Kamarajan Chella, Pandey Ashwini K., Chorlian David B., Zhang Jian, Kuang Weipeng, Pandey Gayathri, et al. 2021. Predicting alcohol use disorder remission: A longitudinal multimodal multi-featured machine learning approach. Translational Psychiatry 11, 1 (2021), 1–10.Google ScholarCross Ref
[70] Kinreich Sivan, Meyers Jacquelyn L., Maron-Katz Adi, Kamarajan Chella, Pandey Ashwini K., Chorlian David B., Zhang Jian, Pandey Gayathri, Viteri Stacey Subbie-Saenz de, Pitti Dan, et al. 2021. Predicting risk for alcohol use disorder using longitudinal data with multimodal biomarkers and family history: A machine learning study. Molecular Psychiatry 26, 4 (2021), 1133–1141.Google ScholarCross Ref
[71] Koehrsen W.. 2018. Hyperparameter Tuning the Random Forest in Python. https://towardsdatascience.com/hyperparameter-tuning-the-random-forest-in-python-using-scikit-learn-28d2aa77dd74.Google Scholar
[72] Lai Tze Leung. 1987. Adaptive treatment allocation and the multi-armed bandit problem. Annals of Statistics 15, 3 (1987), 1091–1114.Google Scholar
[73] Lansford Jennifer E., Dodge Kenneth A., Pettit Gregory S., Bates John E., Crozier Joseph, and Kaplow Julie. 2002. A 12-year prospective study of the long-term effects of early child physical maltreatment on psychological, behavioral, and academic problems in adolescence. Archives of Pediatrics & Adolescent Medicine 156, 8 (2002), 824–830.Google ScholarCross Ref
[74] Lee Brian K., Lessler Justin, and Stuart Elizabeth A.. 2011. Weight trimming and propensity score weighting. PloS One 6, 3 (2011), e18174.Google ScholarCross Ref
[75] Lee Mary R., Sankar Vignesh, Hammer Aaron, Kennedy William G., Barb Jennifer J., McQueen Philip G., and Leggio Lorenzo. 2019. Using machine learning to classify individuals with alcohol use disorder based on treatment seeking status. EClinicalMedicine 12 (2019), 70–78.Google ScholarCross Ref
[76] Lookatch Samantha J., Elledge L. Chris, Anderson Scott, Shorey Ryan C., Stuart Gregory L., and Moore Todd M.. 2017. Cognitive and psychological changes during 28-day residential substance use treatment. Addiction Research & Theory 25, 4 (2017), 334–341.Google ScholarCross Ref
[77] Maklin C.. 2019. Gradient-boosted Decision Tree Algorithm Explained. https://towardsdatascience.com/machine-learning-part-18-boosting-algorithms-gradient-boosting-in-python-ef5ae6965be4.Google Scholar
[78] Mandrekar Jayawant N.. 2010. Receiver operating characteristic curve in diagnostic test assessment. Journal of Thoracic Oncology 5, 9 (2010), 1315–1316.Google ScholarCross Ref
[79] Mate Aditya, Perrault Andrew, and Tambe Milind. 2021. Risk-aware interventions in public health: Planning with restless multi-armed bandits. In 20th International Conference on Autonomous Agents and Multiagent Systems (AAMAS’21). Vol. 10.Google Scholar
[80] McCabe Sean Esteban, West Brady T., Jutkiewicz Emily M., and Boyd Carol J.. 2017. Multiple DSM-5 substance use disorders: A national study of US adults. Human Psychopharmacology: Clinical and Experimental 32, 5 (2017), e2625.Google ScholarCross Ref
[81] McCaffrey Daniel F., Ridgeway Greg, and Morral Andrew R.. 2004. Propensity score estimation with boosted regression for evaluating causal effects in observational studies.Psychological Methods 9, 4 (2004), 403.Google ScholarCross Ref
[82] McCarty Dennis, Braude Lisa, Lyman D. Russell, Dougherty Richard H., Daniels Allen S., Ghose Sushmita Shoma, and Delphin-Rittmon Miriam E.. 2014. Substance abuse intensive outpatient programs: Assessing the evidence. Psychiatric Services 65, 6 (2014), 718–726.Google ScholarCross Ref
[83] McKellar John, Kelly John, Harris Alex, and Moos Rudolf. 2006. Pretreatment and during treatment risk factors for dropout among patients with substance use disorders. Addictive Behaviors 31, 3 (2006), 450–460.Google ScholarCross Ref
[84] McLellan A. Thomas, Lewis David C., O’Brien Charles P., and Kleber Herbert D.. 2000. Drug dependence, a chronic medical illness: Implications for treatment, insurance, and outcomes evaluation. Jama 284, 13 (2000), 1689–1695.Google ScholarCross Ref
[85] McLellan A. Thomas, Woody George E., Luborsky Lester, O’Brien Charles P., and Druley Keith A.. 1983. Increased effectiveness of substance abuse treatment: A prospective study of patient–treatment “matching.”Journal of Nervous and Mental Disease 171, 10 (1983), 597–605.Google ScholarCross Ref
[86] Mee-Lee David, Shulman G. D., Fishman M. J., D. R. Gastfriend, M. M. Miller, and S. M. Provence (Eds.). 2013. The ASAM Criteria: Treatment for Addictive, Substance-related, and Co-occurring Conditions. American Society of Addiction Medicine, Chevy Chase, MD.Google Scholar
[87] Melnick Gerald, Leon George De, Hawke Josephine, Jainchill Nancy, and Kressel David. 1997. Motivation and readiness for therapeutic community treatment among adolescents and adult substance abusers. American Journal of Drug and Alcohol Abuse 23, 4 (1997), 485–506.Google ScholarCross Ref
[88] Merkx Maarten J. M., Schippers Gerard M., Koeter Maarten J. W., Vuijk Pieter Jelle, Oudejans Suzan, Vries Carlijn C. Q. De, and Brink Wim Van Den. 2007. Allocation of substance use disorder patients to appropriate levels of care: Feasibility of matching guidelines in routine practice in Dutch treatment centres. Addiction 102, 3 (2007), 466–474.Google ScholarCross Ref
[89] Mojtabai Ramin and Zivin Joshua Graff. 2003. Effectiveness and cost-effectiveness of four treatment modalities for substance disorders: A propensity score analysis. Health Services Research 38, 1p1 (2003), 233–259.Google ScholarCross Ref
[90] Moos Rudolf H., Pettit Becky, and Gruber Valerie. 1995. Longer episodes of community residential care reduce substance abuse patients’ readmission rates.Journal of Studies on Alcohol 56, 4 (1995), 433–443.Google ScholarCross Ref
[91] Nasir Murtaza, Summerfield Nichalin S., Oztekin Asil, Knight Margaret, Ackerson Leland K., and Carreiro Stephanie. 2021. Machine learning–based outcome prediction and novel hypotheses generation for substance use disorder treatment. Journal of the American Medical Informatics Association 28, 6 (2021), 1216–1224.Google ScholarCross Ref
[92] Negoescu Diana M., Bimpikis Kostas, Brandeau Margaret L., and Iancu Dan A.. 2018. Dynamic learning of patient response types: An application to treating chronic diseases. Management Science 64, 8 (2018), 3469–3488.Google ScholarDigital Library
[93] Nemati S., Ghassemi M. M., and Clifford G. D.. 2016. Optimal medication dosing from suboptimal clinical examples: A deep reinforcement learning approach. In 38th Annual International Conference of IEEE Engineering in Medicine and Biology Society. 2978–2981.Google Scholar
[94] Nordfjaern Trond. 2011. Relapse patterns among patients with substance use disorders. Journal of Substance Use 16, 4 (2011), 313–329.Google ScholarCross Ref
[95] Oommen B. John and Christensen Jens Peter Reus. 1988. Epsilon-optimal discretized linear reward-penalty learning automata. IEEE Transactions on Systems, Man, and Cybernetics 18, 3 (1988), 451–458.Google ScholarCross Ref
[96] Parbhoo S., Bogojeska J., Zazzi M., Roth V., and Doshi-Velez F.. 2017. Combining kernel and model based learning for HIV therapy selection. In AMIA Summits on Translational Science Proceedings. 239–248.Google Scholar
[97] Park Sujeong and Powell David. 2021. Is the rise in illicit opioids affecting labor supply and disability claiming rates?Journal of Health Economics 76 (2021), 102430.Google ScholarCross Ref
[98] Park So Jin, Lee Sun Jung, Kim HyungMin, Kim Jae Kwon, Chun Ji-Won, Lee Soo-Jung, Lee Hae Kook, Kim Dai Jin, and Choi In Young. 2021. Machine learning prediction of dropping out of outpatients with alcohol use disorders. Plos One 16, 8 (2021), e0255626.Google ScholarCross Ref
[99] Pineau Joelle, Guez Arthur, Vincent Robert, Panuccio Gabriella, and Avoli Massimo. 2009. Treating epilepsy via adaptive neurostimulation: A reinforcement learning approach. International Journal of Neural Systems 19, 4 (2009), 227–240.Google ScholarCross Ref
[100] Prasad N., Cheng L., Chivers C., Draugelis M., and Engelhardt B. E.. 2017. A reinforcement learning approach to weaning of mechanical ventilation in intensive care units. arXiv preprint 1704.06300 (2017).Google Scholar
[101] Ptonski P.. 2020. How Many Trees in the Random Forest? https://mljar.com/blog/how-many-trees-in-random-forest/.Google Scholar
[102] Raghu A., Komorowski M., Celi L. A., Szolovits P., and Ghassemi M.. 2017. Continuous state-space models for optimal sepsis treatment-A deep reinforcement learning approach. arXiv preprint 1705.08422 (2017).Google Scholar
[103] Reback Cathy J., Rünger Dennis, Fletcher Jesse B., and Swendeman Dallas. 2018. Ecological momentary assessments for self-monitoring and counseling to optimize methamphetamine treatment and sexual risk reduction outcomes among gay and bisexual men. Journal of Substance Abuse Treatment 92 (2018), 17–26.Google ScholarCross Ref
[104] J. Reid, P. Macchetto, and S. Foster 1999. No Safe Haven: Children of Substance-abusing Parents. New York: Center on Addiction and Sub-stance Abuse.Google Scholar
[105] Roebuck M. Christopher, French Michael T., and McLellan A. Thomas. 2003. DATStats: Results from 85 studies using the drug abuse treatment cost analysis program (DATCAP). Journal of Substance Abuse Treatment 25, 1 (2003), 51–57.Google ScholarCross Ref
[106] Saloner Brendan, McGinty Emma E., Beletsky Leo, Bluthenthal Ricky, Beyrer Chris, Botticelli Michael, and Sherman Susan G.. 2018. A public health strategy for the opioid crisis. Public Health Reports 133, 1_suppl (2018), 24S–34S.Google ScholarCross Ref
[107] Samet Sharon, Fenton Miriam C., Nunes Edward, Greenstein Eliana, Aharonovich Efrat, and Hasin Deborah. 2013. Effects of independent and substance-induced major depressive disorder on remission and relapse of alcohol, cocaine and heroin dependence. Addiction 108, 1 (2013), 115–123.Google ScholarCross Ref
[108] Schmidt Lasse M., Hesse Moten, and Lykke Jørn. 2011. The impact of substance use disorders on the course of schizophrenia–A 15-year follow-up study: dual diagnosis over 15 years. Schizophrenia Research 130, 1–3 (2011), 228–233.Google ScholarCross Ref
[109] Si Nian, Zhang Fan, Zhou Zhengyuan, and Blanchet Jose. 2020. Distributionally robust policy evaluation and learning in offline contextual bandits. In International Conference on Machine Learning. PMLR, 8884–8894.Google Scholar
[110] Sidford Aaron, Wang Mengdi, Wu Xian, Yang Lin F., and Ye Yinyu. 2018. Near-optimal time and sample complexities for solving discounted Markov decision process with a generative model. arXiv preprint arXiv:1806.01492 (2018).Google Scholar
[111] Staiger Petra K., Richardson Ben, Long Caroline M., Carr Victoria, and Marlatt G. Alan. 2013. Overlooked and underestimated? Problematic alcohol use in clients recovering from drug dependence. Addiction 108, 7 (2013), 1188–1193.Google ScholarCross Ref
[112] Stallvik Marianne and Gastfriend David R.. 2014. Predictive and convergent validity of the ASAM criteria software in Norway. Addiction Research & Theory 22, 6 (2014), 515–523.Google ScholarCross Ref
[113] Stallvik Marianne, Gastfriend David R., and Nordahl Hans M.. 2015. Matching patients with substance use disorder to optimal level of care with the ASAM criteria software. Journal of Substance Use 20, 6 (2015), 389–398.Google ScholarCross Ref
[114] Tao Yebin, Wang Lu, and Almirall Daniel. 2018. Tree-based reinforcement learning for estimating optimal dynamic treatment regimes. Annals of Applied Statistics 12, 3 (2018), 1914.Google ScholarCross Ref
[115] Utomo C. P., Kurniawati H., Li X., and Pokharel S.. 2019. Personalized Medicine in Critical Care Using Bayesian Reinforcement Learning. Springer, 648–657.Google Scholar
[116] Vermorel Joannes and Mohri Mehryar. 2005. Multi-armed bandit algorithms and empirical evaluation. In European Conference on Machine Learning. Springer, 437–448.Google ScholarDigital Library
[117] Vigo Daniel V., Kestel Devora, Pendakur Krishna, Thornicroft Graham, and Atun Rifat. 2019. Disease burden and government spending on mental, neurological, and substance use disorders, and self-harm: Cross-sectional, ecological study of health system response in the Americas. Lancet Public Health 4, 2 (2019), e89–e96.Google ScholarCross Ref
[118] Wadekar Adway. 2019. Predicting opioid use disorder (OUD) using a random forest. In 2019 IEEE 43rd Annual Computer Software and Applications Conference (COMPSAC’19), Vol. 1. IEEE, 960–961.Google ScholarCross Ref
[119] Wang Yingfei and Powell Warren. 2016. An optimal learning method for developing personalized treatment regimes. arXiv preprint arXiv:1607.01462 (2016).Google Scholar
[120] Watts J., Khojandi A., Vasudevan R., and Ramdhani R.. 2020. Optimizing individualized treatment planning for Parkinson’s disease using deep reinforcement learning. In 2020 42nd Annual International Conference of the IEEE Engineering in Medicine & Biology Society (EMBC’20). IEEE, 5406–5409.Google ScholarCross Ref
[121] Williamson Anna, Darke Shane, Ross Joanne, and Teesson Maree. 2006. The effect of persistence of cocaine use on 12-month outcomes for the treatment of heroin dependence. Drug and Alcohol Dependence 81, 3 (2006), 293–300.Google ScholarCross Ref
[122] Xie Haiyi, McHugo Gregory J., Fox Melinda B., and Drake Robert E.. 2005. Special section on relapse prevention: Substance abuse relapse in a ten-year prospective follow-up of clients with mental and substance use disorders. Psychiatric Services 56, 10 (2005), 1282–1287.Google ScholarCross Ref
[123] Xie Haiyi, McHugo Gregory J., Fox Melinda B., and Drake Robert E.. 2005. Special section on relapse prevention: Substance abuse relapse in a ten-year prospective follow-up of clients with mental and substance use disorders. Psychiatric Services 56, 10 (2005), 1282–1287.Google ScholarCross Ref
[124] Zanarini Mary C., Frankenbur Frances R., Weingeroff Jolie L., Reich D. Bradford, Fitzmaurice Garrett M., and Weiss Roger D.. 2011. The course of substance use disorders in patients with borderline personality disorder and Axis II comparison subjects: A 10-year follow-up study. Addiction 106, 2 (2011), 342–348.Google ScholarCross Ref
[125] Zhang Zhiwei, Friedmann Peter D., and Gerstein Dean R.. 2003. Does retention matter? Treatment duration and improvement in drug use. Addiction 98, 5 (2003), 673–684.Google ScholarCross Ref
[126] Zhang-James Yanli, Chen Qi, Kuja-Halkola Ralf, Lichtenstein Paul, Larsson Henrik, and Faraone Stephen V.. 2020. Machine-Learning prediction of comorbid substance use disorders in ADHD youth using Swedish registry data. Journal of Child Psychology and Psychiatry 61, 12 (2020), 1370–1379.Google ScholarCross Ref
[127] Zhou Zhijin, Wang Yingfei, Mamani Hamed, and Coffey David G.. 2019. How do tumor cytogenetics inform cancer treatments? Dynamic risk stratification and precision medicine using multi-armed bandits. In Dynamic Risk Stratification and Precision Medicine Using Multi-armed Bandits.Google Scholar

Index Terms

Optimizing Substance Use Treatment Selection Using Reinforcement Learning
1. Applied computing
  1. Life and medical sciences
    1. Health care information systems
    2. Health informatics
2. Computing methodologies
  1. Machine learning
    1. Learning paradigms
      1. Reinforcement learning
    2. Machine learning approaches
      1. Classification and regression trees

Recommendations

Cyberbullying victimization and substance use among Quebec high schools students: The mediating role of psychological distress
Abstract
Cyberbullying has become a significant public health issue among youth and is associated with numerous mental health problems. While the majority of studies explored its mental consequences using cross-sectional design, this article ...
Highlights
- Cyberbullying victimization, exposure to family violence and gender are significantly associated to psychological distress.
Read More
Reward Shaping in Episodic Reinforcement Learning
AAMAS '17: Proceedings of the 16th Conference on Autonomous Agents and MultiAgent Systems

Recent advancements in reinforcement learning confirm that reinforcement learning techniques can solve large scale problems leading to high quality autonomous decision making. It is a matter of time until we will see large scale applications of ...
Read More
Relational Reinforcement Learning

Relational reinforcement learning is presented, a learning technique that combines reinforcement learning with relational learning or inductive logic programming. Due to the use of a more expressive representation language to represent states, actions ...
Read More

Comments

Login options

Check if you have access through your login credentials or your institution to get full access on this article.

Full Access

Get this Article

Published in
ACM Transactions on Management Information Systems Volume 14, Issue 2
June 2023
178 pages
ISSN:2158-656X
EISSN:2158-6578
DOI:10.1145/3580448
Editor:
Daniel Zeng
Chinese Academy of Sciences, China
Issue’s Table of Contents
Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than ACM must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected].
Sponsors
In-Cooperation
Publisher
Association for Computing Machinery
New York, NY, United States
Publication History
- Published: 25 March 2023
- Online AM: 16 September 2022
- Accepted: 3 September 2022
- Revised: 16 June 2022
- Received: 14 January 2022
Published in tmis Volume 14, Issue 2

Permissions
Request permissions about this article.
Request Permissions

Check for updates
Author Tags
Contextual bandits
substance use
reinforcement learning
Qualifiers
- research-article
Conference
Funding Sources
Other Metrics
View Article Metrics

Article Metrics
- 0
  Total Citations
  View Citations
- 459
  Total Downloads
- Downloads (Last 12 months)282
- Downloads (Last 6 weeks)13
Other Metrics
View Author Metrics
Cited By
This publication has not been cited yet

PDF Format

View or Download as a PDF file.

PDF

eReader

View online with eReader.

eReader

Full Text

View this article in Full Text.

View Full Text

HTML Format

View this article in HTML Format .

View HTML Format

Optimizing Substance Use Treatment Selection Using Reinforcement Learning

ACM Transactions on Management Information Systems

Abstract

REFERENCES

Cited By

Index Terms

Recommendations

Cyberbullying victimization and substance use among Quebec high schools students: The mediating role of psychological distress

Reward Shaping in Episodic Reinforcement Learning

Relational Reinforcement Learning