Coordinating patient preferences through automated negotiation: A multiagent systems model for diagnostic services scheduling
Introduction
Effective management of patient preferences in medical appointment scheduling has benefits that go beyond simply improving patient satisfaction. It contributes to the reduction of unexpected no-shows and cancellations, thus improve the utilization of healthcare resources [1], [2], [3], [4]. As no-shows and cancellations can generate substantial financial costs to healthcare systems [5], [6], [7], accommodating patient preferences in appointment scheduling can have direct impacts on hospitals’ bottom line. This is particularly true in diagnostic imaging services as the facilities required, such as magnetic resonance imaging (MRI) scanning or computed tomography (CT) scanning installations, are expensive assets in hospitals and these services are in constant high demand [8]. To this end, effective utilization of these facilities plays a significant role in improving both patient satisfaction and hospital operating efficiency [3], [9].
In hospitals, patients who require diagnostic services are usually classified into two categories: inpatients and outpatients. The inpatient demand has higher priority and must be met the day it arrives. The low priority outpatient demand is usually booked prior to knowing the high priority demand for that day [10]. To ensure sufficient resources for high priority demand, hospitals have to reserve a fixed portion of daily capacity for inpatients leaving little room for outpatients [11], [12]. In addition, unexpected no-shows and last minute cancellations from outpatients further jeopardize the utilization of already limited resources, which results in excessive waiting times in healthcare systems. According to an annual report [13] produced by The Fraser Institute (the top Canadian think tank based on Vancouver, British Columbia), in 2017, Canadian patients faced the longest ever wait time for medically necessary treatment in more than two decades of tracking. The median diagnostic services waiting time is about 4.1 weeks for a CT scan, 10.8 weeks for an MRI and 3.9 weeks for an ultrasound, which are significantly longer than recommended. To reduce waiting times, the report suggests that Canadian public healthcare systems can acquire new capacities by partnering with private medical facilities. However, in addition to acquiring new capacities, it is equally important to improve the utilization of the existing public healthcare capacities by reducing unexpected no-shows and cancellations through effective appointment scheduling.
There has been growing research efforts on outpatient appointment scheduling in the past two decays. Comprehensive reviews of the literature before 2008 can be found in Cayirli et al. [14] and Gupta et al. [9]. A recent review by Amir et al. [15] classifies the outpatient appointment scheduling literature into three categories based on the level of decision making, namely strategic level, tactic level and operational level. Long-term decisions, such as access policies [3], [16], and resources planning [17], which determine appointment scheduling systems structures are considered as strategic decisions. At tactic level, decision making focuses on the allocation of appointment capacity to different patient groups or how patients as a whole are scheduled [10]. Decisions on how to schedule individual patients are considered as operational level decisions. Typical examples of such decisions include scheduling patients to servers/resources in settings with multiple identical service providers [18], [19] or different service providers [20] and selecting patients from waiting lists [11].
At all three levels of decision making, uncertainties are considered as the main challenge in developing effective scheduling [2], [16], [10]. At operational level, these uncertainties are mainly caused by environmental factors such as unpunctuality, no-show, cancellation and random service time [11]. Stochastic programming methods such as single-stage [21] and two-stage [22] stochastic programming have been proposed as optimization modeling approaches for dealing with uncertainties at this level. In on-line settings, Markov decision process is the most useful stochastic dynamic programming approach [2].
Existing stochastic optimization and stochastic dynamic programming approaches are reactive in the sense that they assume the uncertainties are uncontrollable and try to develop models to optimize appointment system efficiency given the seemingly extraneous uncertainties. In this paper, we take a proactive approach which reduces the uncertainties from the source by incorporating patient preferences into the appointment scheduling process. The idea is supported by the studies in [2], [3], [4], which all point out that accommodating patient preferences in appointment scheduling can reduce the number of no-shows, unpunctuality and cancellation, thereby increase the operational efficiency and resource utilization.
Scheduling decisions with the consideration of patient preferences have received limited attention in the appointment scheduling literature. In [3], Gupta et al. presented an appointment scheduling model which allows patients to select their preferred time slots and physicians. They further developed an adaptive appointment scheduling system in [2], which dynamically learns and updates patients’ preferences. The Online Patient Scheduling model proposed in [28] focuses on a simple Boolean type preference. Patients are scheduled either to their preferred time slots or to non-preferred time slots. Feldman et al. [2] focus on patient choice between multiple appointment days. They apply a multinomial logit model to govern the patient preferences. In this model, each patient associates a preference weight to future appointment days and a nominal preference weight of one indicates an undesirable day for the patient. In [4], a Markov decision process model is proposed with the consideration of patient preferences for patient online booking. In order to learn and update patients preferences, an adaptive programming approach is adopted.
The above mentioned models that accommodate patients’ preferences are mainly operations research based approaches with the assumption of centralized hospital environment in which the resource allocation and scheduling decisions are made by a central scheduler. However, hospitals often show a decentralized organizational structure which are characterized by autonomous and independent decision makers [23], which are suitable for Multiagent Systems (MAS) modeling. In [24], schedules of shared CT scanning resources are created collectively through negotiation among different departments in the hospital. Since the departments have their own (medical) priorities and preferences, the proposed negotiation protocol takes strategic behaviors into consideration. MAS modeling are also suitable for the settings where patients with their private information compete for shared hospital resources [12]. In [25], a non-cooperative game is formulated to improve patient hospital choice with the consideration of incomplete patient preferences. A pure Nash equilibrium is calculated as the best prediction of the patients’ choices of hospitals. In [26], Maartje et al. study how to motivate hospital departments to reveal their true demands by modeling the MRI scan capacity allocation problem as a Bayesian game. They prove that truthful telling is a Bayesian Nash equilibrium.
Paulussen et al. [27] present a multiagent system to support patient scheduling. A market based mechanism is designed to facilitate the coordination among patient agents to improve the overall schedule quality. Auctions have also been proposed for appointment scheduling. Grace et al. [28] propose a multiagent framework in which patient and hospital resources are represented as agents who participate in a combinatorial auction which determines the schedule through winner determination. In their model, patient agents first submit their feasible time slots in accordance with their availability and preferences. After that each resource agent calculates the initial prices for various time slots based on the sum of the weighted waiting time of all the patients from the feasible schedule. Final schedule is calculated with the objective of minimizing the sum of patients’ waiting times. In [29], [30], iterative auctions are used for agent-based appointment scheduling where patient agents bid for their preferred time slots. In [31], a first-price sealed-bid auction is designed for decentralized patient scheduling with the aim of maximizing patient preferences.
In this paper, we assume a MAS environment in which patients are modeled as autonomous and self-interested agents. Compared with traditional centralized scheduling models, MAS scheduling poses additional challenges attributable to its decentralized nature. Traditionally, scheduling problems are studied under an assumption that an algorithm has access to all the information (or at least the statistic distribution of the information in the non-deterministic cases) needed to compute a schedule. However, in MAS where self-interested agents are involved, an algorithm may not have access to all the needed information because the information is privately held by the agents who cannot be assumed to follow the algorithm but rather their own self-interests. In this decentralized setting, agents behave strategically in the pursuit of their individual objectives rather than system wide optimality, which calls for game theory-based scheduling approaches [32], [33], [34]. When designing the game theory-based scheduling mechanism for MAS, two properties of the mechanism are of importance, namely individual rationality (IR) and incentive compatibility (IC) [35]. IR ensures that the utilities of all participants are not negative such that independent agents cannot lose by participating, which provides them incentives to be part of the mechanism. On the other hand, IC ensures that an agent’s (weakly) dominant strategy [35] is to truthfully report their private information. In our case, this property motivates patients to truthfully submit their availabilities.
The contribution of this paper can be seen from three perspectives. From scheduling optimization perspective, we present a MAS scheduling mechanism for diagnostic services scheduling problems in game theoretic settings, which can not be solved by traditional centralized scheduling algorithms. The proposed mechanism possesses both IR and IC properties. It is also efficient for larger problem instances. Secondly, from mechanism design perspective, compared with existing mutiagent systems appointment scheduling models, the uniqueness of this work is that the negotiation among patients is conducted through a non-price mechanism, which is desirable in healthcare settings in general and, in many cases, required in public healthcare systems. Our contributions from scheduling and mechanism design perspectives lead to a unique scheduling system which brings managerial benefits to hospitals: the proposed approach strike the balance between patients’ priorities and their time preferences which improves overall patients’ welfare and satisfaction.
The rest of this paper is organized as follows. In Section 2, a formulation of the patient diagnostic service scheduling problem is provided and a multiagent systems architecture is proposed. In Section 3, negotiation protocols for patient selection problem and preference scheduling problem are presented. In Section 4, we evaluate the performance of the proposed scheduling approach through a computational study. We conclude the paper and discuss future work in Section 5.
Section snippets
Diagnostic services scheduling
We focus on an outpatient diagnostic services scheduling setting, where there is a fairly large number of patients waiting in the queue for diagnostic imaging services. To reduce the impact of future uncertainties on service schedules, the hospital takes a rolling horizon scheduling approach by periodically making a time window (say a week for example) available for scheduling a portion of patients in the queue. We assume each patient has preferences on multiple time horizons. There are two
Negotiation protocols
Within the architecture of the multiagent systems model, diagnostic services scheduling can be seen as a coordination process between the DS agent and a group of PA agents, during which service time slots are allocated to patients through automated negotiation. The process consists of two stages, namely patient selection (PSL) and preference scheduling (PSC). A negotiation protocol is used in each of the stages. This process repeats and continues until all the waiting patients are allocated. A
Computational study
We have proposed two negotiation protocols for the multiagent systems model. At the patient selection stage, a contract-net protocol is designed to select patients to be scheduled in the available time window. The group of selected patients are computed through the construction of an initial allocation which maximizes overall priority levels. Given the individually rational and incentive compatible properties of the protocol, the DS agent can obtain optimal solutions by solving the PSL model
Conclusion
Accommodating patient preferences in diagnostic services scheduling reduces no-show and cancellation rates resulting in improved utilization of expensive diagnostic imaging resources. Given inherently decentralized hospital environments and patients’ strategic behaviors, we have proposed a multiagent systems model for allocating diagnostic service time slots to outpatients through automated negotiation. The proposed negotiation protocol serves as a coordination mechanism for patients to
Declaration of Competing Interest
The authors declared that there is no conflict of interest.
References (43)
- et al.
Outpatient appointment systems in healthcare: A review of optimization studies
Eur. J. Oper. Res.
(2017) - et al.
Matching daily healthcare provider capacity to demand in advanced access scheduling systems
Eur. J. Oper. Res.
(2007) - et al.
Algorithms for scheduling of chemotherapy plans
Comput. Biol. Med.
(2013) - et al.
A markov decision process approach to multi-category patient scheduling in a diagnostic facility
Artif. Intell. Med.
(2011) - et al.
Optimal booking and scheduling in outpatient procedure centers
Comput. Oper. Res.
(2014) - et al.
Optimal allocation of mri scan capacity among competing hospital departments
Eur. J. Oper. Res.
(2012) - et al.
Agents applied in health care: A review
Int. J. Med. Informat.
(2010) - et al.
Game theory: applications for surgeons and the operating room environment
Surgery
(2012) - et al.
Optimizing agent-based meeting scheduling through preference estimation
Eng. Appl. Artif. Intell.
(2003) - et al.
Design of automated negotiation mechanisms for decentralized heterogeneous machine scheduling
Eur. J. Oper. Res.
(2016)
Negotiation-based collaborative planning between supply chains partners
Eur. J. Oper. Res.
Improving health care by understanding patient preferences: the role of computer technology
J. Am. Med. Inform. Assoc.
Appointment scheduling under patient preference and no-show behavior
Oper. Res.
Revenue management for a primary-care clinic in the presence of patient choice
Oper. Res.
Adaptive appointment systems with patient preferences
Manuf. Serv. Oper. Manage.
Reducing delays for medical appointments: A queueing approach
Oper. Res.
Appointment scheduling with no-shows and overbooking
Prod. Oper. Manage.
Dynamic scheduling of outpatient appointments under patient no-shows and cancellations
Manuf. Serv. Oper. Manage.
An efficient outpatient scheduling approach
IEEE Trans. Automat. Sci. Eng.
Appointment scheduling in health care: Challenges and opportunities
IIE Trans.
Improving resource utilization for diagnostic services through flexible inpatient scheduling: A method for improving resource utilization
J. Oper. Res. Soc.
Cited by (9)
Optimal production scheduling with multi-round information interaction for demander-dominated decentralized scheduling problem
2023, Engineering Applications of Artificial IntelligenceSocial welfare maximizing fleet charging scheduling through voting-based negotiation
2021, Transportation Research Part C: Emerging TechnologiesCitation Excerpt :To overcome this, the proposed voting mechanism implements a decentralized improvement-based searching process which is inspired by the simulated annealing (SA) meta-heuristic. In addition, we do not assume a given preference value for each of the voting participants as in Lang et al. (2016, 2019). Instead, we design a driver-oriented valuation function to support the negotiation process and to concisely express drivers’ charging preferences.
A context-aware approach to automated negotiation using reinforcement learning
2021, Advanced Engineering InformaticsCitation Excerpt :In [5], the existence of a communication channel between involved agents provides the basis for negotiation and the analysis of the outcomes obtained by selfish and pro-social agents. In [19], a decision support system for human negotiation is detailed, whereas in [12], automated negotiations are used for scheduling of patient diagnostic services. In [8], negotiation among agents that are capable of using different orders of theory of mind (the ability to reason explicitly about unknown mental states of others) is proposed.
A Machine Learning-Based Approach for Multi-AGV Dispatching at Automated Container Terminals
2023, Journal of Marine Science and EngineeringVOMA: A Privacy-Preserving Matching Mechanism Design for Community Ride-Sharing
2022, IEEE Transactions on Intelligent Transportation Systems