Abstract
We propose a prescriptive learning approach for revenue management in air-cargo that combines machine learning prediction with decision making using deep reinforcement learning. This approach, named RL-Cargo, addresses a problem that is unique to the air-cargo business, namely the wide discrepancy between the quantity (weight or volume) that a shipper will book and the actual amount received at departure time by the airline. The discrepancy results in sub-optimal and inefficient behavior by both the shipper and the airline resulting in an overall loss of potential revenue for the airline. In the proposed approach, booking features and extracted disguised missing values are exploited to provide a prediction on the received volume, while a DQN method using uncertainty bounds from the prediction intervals is proposed for decision making. We have validated the benefits of RL-Cargo using a real dataset of 1000 flights to compare classical Dynamic Programming and Deep Reinforcement Learning techniques on offloading costs and revenue generation. Our results suggest that prescriptive learning which combines prediction with decision making provides a principled approach for managing the air cargo revenue ecosystem. Furthermore, the proposed approach can be abstracted to many other application domains where decision making needs to be carried out in face of both data and behavioral uncertainty.
This is a preview of subscription content, log in via an institution to check access.
Similar content being viewed by others
Notes
We omit the company name for confidentiality reasons.
We used a threshold of 0.01%.
For non-cargo flights, \(k_v\) varies depending upon passenger load.
We have overloaded the \(f_\theta \) signature to emphasize the role of bkvol.
It is S(T, m), Stirling number of second kind.
References
Popescu A (2006) Air cargo revenue and capacity management. Georgia Institute of Technology
Boonekamp, Thijs and Gromicho, J and Dullaert, W and Radstaak, B (2013) Unpublished masters thesis, Vrije Universiteit, Amsterdam
Amaruchkul K, Cooper WL, Gupta D (2007) Single-leg air-cargo revenue management. Trans Sci INFORMS 41:457–469
Chiang W-C, Chen JC, Xiaojing X (2007) Single-leg air-cargo revenue management. An overview of research on revenue management: current issues and future research. Int J Revenue Manag (IJRM) 1:97
Mnih V, et al (2013) Uncertainty-aware action advising for deep reinforcement learning agents. arXiv preprint arXiv:1312.5602
Da Silva FL, Hernandez-Leal P, Kartal B, Taylor ME (2020) Uncertainty-aware action advising for deep reinforcement learning agents. In: Proceedings of the AAAI Conference on Artificial Intelligence, pp 5792–5799
ODonoghue B, Osband I, Munos R, Mnih V (2018) The uncertainty bellman equation and exploration. Int Conf Mach Learn pp 3836–3845
Kahn G, Villaflor A, Pong V, Abbeel P, Levine S (2017) Online evaluation of bid prediction models in a Uncertainty-aware reinforcement learning for collision avoidance. arXiv preprint arXiv:1702.01182
Larsen T, Leslie DS, Collins EJ, Bogacz R (2010) Posterior weighted reinforcement learning with state uncertainty. MIT Press, Neural computation, pp 1149–1179
McGill JI, Van Ryzin GJ (1999) Revenue management: research overview and prospects. Trans Sci 33:233–256
Hueglin C, Vannotti F (2001) Data mining techniques to improve forecast accuracy in airline business. In: Proceedings of the Seventh ACM SIGKDD International Conference on Knowledge Discovery and Data Mining, pp 438–442
Lan Y, Ball MO, Karaesmen IZ (2011) Regret in overbooking and fare-class allocation for single leg. Manuf Serv Oper Manag INFORMS 13:194–208
Popescu A, Keskinocak P, Johnson E, LaDue M, Kasilingam R (2006) Estimating air-cargo overbooking based on a discrete show-up-rate distributiong. Interfaces INFORMS 36:248–258
Kasilingam RG (1997) Air cargo revenue management: characteristics and complexities. Transportation science, INFORMS, pp 457–469
S Budiarto, HP Putro, P Pradono, G Yudok (2018) Revenue management of air cargo service in theory and practice. In: IOP Conf Ser Earth Environ Sci pp 12–22
Shariat S, Orten B, Dasdan A (2017) Online evaluation of bid prediction models in a large-scale computational advertising platform: decision making and insights. Knowledge and Information Systems, Springer, pp 37–60
Larsen T, Leslie DS, Collins EJ, Bogacz R (2017) Supervised item response models for informative prediction. Knowledge and Information Systems, Springer, pp 235–257
Sutton RS, Barto AG (2018) Reinforcement learning: an introduction. MIT press
Powell W (2007) Approximate dynamic programming: solving the curses of dimensionalitys. Wiley
Bertsekas DP, Bertsekas DP, Bertsekas DP, Bertsekas D (2005) Dynamic programming and optimal control. Athena scientific Belmont, MA
Vlassis N, Ghavamzadeh M, Mannor S, Poupart Pascal (2012) Bayesian reinforcement learning. Reinforcement learning, pp 359–386
Raman V, Hellerstein JM (2001) Potter’s wheel: An interactive data cleaning system. VLDB 1:381–390
Raman V, Hellerstein JM (2007) Duplicate record detection: a survey. IEEE Trans Knowl Data Eng 19:1–16
Pearson RK (2006) Revenue management of air cargo service in theory and practice. SIGKDD Explor Newsl pp 83–92
Pearson RK (2007) Cleaning disguised missing data: a heuristic approach. In: Proceedings of the 13th ACM SIGKDD International Conference on Knowledge Discovery and Data Mining, pp 950–958
Qahtan AA, and Elmagarmid A, Castro Fernandez R, Ouzzani M, Tang N (2018) FAHES: a robust disguised missing values detector. In: Proceedings of the 24th ACM SIGKDD International Conference on Knowledge Discovery and Data Mining, ACM, 2018, pp 2100–2109
Slager B, Kapteijns L (2004) Implementation of cargo revenue management at KLM. J Revenue Pricing Manag 3:80–90
Beidermand D (2002) New freight dimensions: For shippers, finding the real cost of a shipment shouldn’t be a matter of weight and see. Air Cargo World, Washington, pp 34–40
Chen T, Guestrin C (2016) XGBoost: A Scalable Tree Boosting System. In: Proceedings of the 22Nd ACM SIGKDD International Conference on Knowledge Discovery and Data Mining. ACM, 2016, pp 785–794
Tibshirani TR, Friedman J (2009) Elements of atatistical learning. Springer, pp 643–651
Richard B (1957) Q-learning. J Math Mech
Richard S, Andrew GB (2018) Reinforcement learning: an introduction. MIT press
Watkins CJ, Dayan P (1992) Q-learning. Mach Learn pp 279–29
Author information
Authors and Affiliations
Corresponding author
Additional information
Publisher's Note
Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.
Work carried out in Qatar Computing Research Institute, prior to joining Amazon.
Rights and permissions
Springer Nature or its licensor holds exclusive rights to this article under a publishing agreement with the author(s) or other rightsholder(s); author self-archiving of the accepted manuscript version of this article is solely governed by the terms of such publishing agreement and applicable law.
About this article
Cite this article
Rizzo, S.G., Chen, Y., Pang, L. et al. Uncertainty-bounded reinforcement learning for revenue optimization in air cargo: a prescriptive learning approach. Knowl Inf Syst 64, 2515–2541 (2022). https://doi.org/10.1007/s10115-022-01713-5
Received:
Revised:
Accepted:
Published:
Issue Date:
DOI: https://doi.org/10.1007/s10115-022-01713-5