Coordinating patient preferences through automated negotiation: A multiagent systems model for diagnostic services scheduling

Abstract

This paper presents a multiagent systems model for patient diagnostic services scheduling. We assume a decentralized environment in which patients are modeled as self-interested agents who behave strategically to advance their own benefits rather than the system wide performance. The objective is to improve the utilization of diagnostic imaging resources by coordinating patient individual preferences through automated negotiation. The negotiation process consists of two stages, namely patient selection and preference scheduling. The contract-net protocol and simulated annealing based meta-heuristics are used to design negotiation protocols at the two stages respectively. In terms of game theoretic properties, we show that the proposed protocols are individually rational and incentive compatible. The performance of the preference scheduling protocol is evaluated by a computational study. The average percentage gap analysis of various configurations of the protocol shows that the results obtained from the protocol are close to the optimal ones. In addition, we present the algorithmic properties of the preference scheduling protocol through the validation of a set of eight hypotheses.

Introduction

Effective management of patient preferences in medical appointment scheduling has benefits that go beyond simply improving patient satisfaction. It contributes to the reduction of unexpected no-shows and cancellations, thus improve the utilization of healthcare resources [1], [2], [3], [4]. As no-shows and cancellations can generate substantial financial costs to healthcare systems [5], [6], [7], accommodating patient preferences in appointment scheduling can have direct impacts on hospitals’ bottom line. This is particularly true in diagnostic imaging services as the facilities required, such as magnetic resonance imaging (MRI) scanning or computed tomography (CT) scanning installations, are expensive assets in hospitals and these services are in constant high demand [8]. To this end, effective utilization of these facilities plays a significant role in improving both patient satisfaction and hospital operating efficiency [3], [9].

In hospitals, patients who require diagnostic services are usually classified into two categories: inpatients and outpatients. The inpatient demand has higher priority and must be met the day it arrives. The low priority outpatient demand is usually booked prior to knowing the high priority demand for that day [10]. To ensure sufficient resources for high priority demand, hospitals have to reserve a fixed portion of daily capacity for inpatients leaving little room for outpatients [11], [12]. In addition, unexpected no-shows and last minute cancellations from outpatients further jeopardize the utilization of already limited resources, which results in excessive waiting times in healthcare systems. According to an annual report [13] produced by The Fraser Institute (the top Canadian think tank based on Vancouver, British Columbia), in 2017, Canadian patients faced the longest ever wait time for medically necessary treatment in more than two decades of tracking. The median diagnostic services waiting time is about 4.1 weeks for a CT scan, 10.8 weeks for an MRI and 3.9 weeks for an ultrasound, which are significantly longer than recommended. To reduce waiting times, the report suggests that Canadian public healthcare systems can acquire new capacities by partnering with private medical facilities. However, in addition to acquiring new capacities, it is equally important to improve the utilization of the existing public healthcare capacities by reducing unexpected no-shows and cancellations through effective appointment scheduling.

There has been growing research efforts on outpatient appointment scheduling in the past two decays. Comprehensive reviews of the literature before 2008 can be found in Cayirli et al. [14] and Gupta et al. [9]. A recent review by Amir et al. [15] classifies the outpatient appointment scheduling literature into three categories based on the level of decision making, namely strategic level, tactic level and operational level. Long-term decisions, such as access policies [3], [16], and resources planning [17], which determine appointment scheduling systems structures are considered as strategic decisions. At tactic level, decision making focuses on the allocation of appointment capacity to different patient groups or how patients as a whole are scheduled [10]. Decisions on how to schedule individual patients are considered as operational level decisions. Typical examples of such decisions include scheduling patients to servers/resources in settings with multiple identical service providers [18], [19] or different service providers [20] and selecting patients from waiting lists [11].

At all three levels of decision making, uncertainties are considered as the main challenge in developing effective scheduling [2], [16], [10]. At operational level, these uncertainties are mainly caused by environmental factors such as unpunctuality, no-show, cancellation and random service time [11]. Stochastic programming methods such as single-stage [21] and two-stage [22] stochastic programming have been proposed as optimization modeling approaches for dealing with uncertainties at this level. In on-line settings, Markov decision process is the most useful stochastic dynamic programming approach [2].

Existing stochastic optimization and stochastic dynamic programming approaches are reactive in the sense that they assume the uncertainties are uncontrollable and try to develop models to optimize appointment system efficiency given the seemingly extraneous uncertainties. In this paper, we take a proactive approach which reduces the uncertainties from the source by incorporating patient preferences into the appointment scheduling process. The idea is supported by the studies in [2], [3], [4], which all point out that accommodating patient preferences in appointment scheduling can reduce the number of no-shows, unpunctuality and cancellation, thereby increase the operational efficiency and resource utilization.

Scheduling decisions with the consideration of patient preferences have received limited attention in the appointment scheduling literature. In [3], Gupta et al. presented an appointment scheduling model which allows patients to select their preferred time slots and physicians. They further developed an adaptive appointment scheduling system in [2], which dynamically learns and updates patients’ preferences. The Online Patient Scheduling model proposed in [28] focuses on a simple Boolean type preference. Patients are scheduled either to their preferred time slots or to non-preferred time slots. Feldman et al. [2] focus on patient choice between multiple appointment days. They apply a multinomial logit model to govern the patient preferences. In this model, each patient associates a preference weight to future appointment days and a nominal preference weight of one indicates an undesirable day for the patient. In [4], a Markov decision process model is proposed with the consideration of patient preferences for patient online booking. In order to learn and update patients preferences, an adaptive programming approach is adopted.

The above mentioned models that accommodate patients’ preferences are mainly operations research based approaches with the assumption of centralized hospital environment in which the resource allocation and scheduling decisions are made by a central scheduler. However, hospitals often show a decentralized organizational structure which are characterized by autonomous and independent decision makers [23], which are suitable for Multiagent Systems (MAS) modeling. In [24], schedules of shared CT scanning resources are created collectively through negotiation among different departments in the hospital. Since the departments have their own (medical) priorities and preferences, the proposed negotiation protocol takes strategic behaviors into consideration. MAS modeling are also suitable for the settings where patients with their private information compete for shared hospital resources [12]. In [25], a non-cooperative game is formulated to improve patient hospital choice with the consideration of incomplete patient preferences. A pure Nash equilibrium is calculated as the best prediction of the patients’ choices of hospitals. In [26], Maartje et al. study how to motivate hospital departments to reveal their true demands by modeling the MRI scan capacity allocation problem as a Bayesian game. They prove that truthful telling is a Bayesian Nash equilibrium.

Paulussen et al. [27] present a multiagent system to support patient scheduling. A market based mechanism is designed to facilitate the coordination among patient agents to improve the overall schedule quality. Auctions have also been proposed for appointment scheduling. Grace et al. [28] propose a multiagent framework in which patient and hospital resources are represented as agents who participate in a combinatorial auction which determines the schedule through winner determination. In their model, patient agents first submit their feasible time slots in accordance with their availability and preferences. After that each resource agent calculates the initial prices for various time slots based on the sum of the weighted waiting time of all the patients from the feasible schedule. Final schedule is calculated with the objective of minimizing the sum of patients’ waiting times. In [29], [30], iterative auctions are used for agent-based appointment scheduling where patient agents bid for their preferred time slots. In [31], a first-price sealed-bid auction is designed for decentralized patient scheduling with the aim of maximizing patient preferences.

In this paper, we assume a MAS environment in which patients are modeled as autonomous and self-interested agents. Compared with traditional centralized scheduling models, MAS scheduling poses additional challenges attributable to its decentralized nature. Traditionally, scheduling problems are studied under an assumption that an algorithm has access to all the information (or at least the statistic distribution of the information in the non-deterministic cases) needed to compute a schedule. However, in MAS where self-interested agents are involved, an algorithm may not have access to all the needed information because the information is privately held by the agents who cannot be assumed to follow the algorithm but rather their own self-interests. In this decentralized setting, agents behave strategically in the pursuit of their individual objectives rather than system wide optimality, which calls for game theory-based scheduling approaches [32], [33], [34]. When designing the game theory-based scheduling mechanism for MAS, two properties of the mechanism are of importance, namely individual rationality (IR) and incentive compatibility (IC) [35]. IR ensures that the utilities of all participants are not negative such that independent agents cannot lose by participating, which provides them incentives to be part of the mechanism. On the other hand, IC ensures that an agent’s (weakly) dominant strategy [35] is to truthfully report their private information. In our case, this property motivates patients to truthfully submit their availabilities.

The contribution of this paper can be seen from three perspectives. From scheduling optimization perspective, we present a MAS scheduling mechanism for diagnostic services scheduling problems in game theoretic settings, which can not be solved by traditional centralized scheduling algorithms. The proposed mechanism possesses both IR and IC properties. It is also efficient for larger problem instances. Secondly, from mechanism design perspective, compared with existing mutiagent systems appointment scheduling models, the uniqueness of this work is that the negotiation among patients is conducted through a non-price mechanism, which is desirable in healthcare settings in general and, in many cases, required in public healthcare systems. Our contributions from scheduling and mechanism design perspectives lead to a unique scheduling system which brings managerial benefits to hospitals: the proposed approach strike the balance between patients’ priorities and their time preferences which improves overall patients’ welfare and satisfaction.

The rest of this paper is organized as follows. In Section 2, a formulation of the patient diagnostic service scheduling problem is provided and a multiagent systems architecture is proposed. In Section 3, negotiation protocols for patient selection problem and preference scheduling problem are presented. In Section 4, we evaluate the performance of the proposed scheduling approach through a computational study. We conclude the paper and discuss future work in Section 5.