High performance risk aggregation: addressing the data processing challenge the hadoop mapreduce way
Pages 53 - 60
Abstract
Monte Carlo simulations employed for the analysis of portfolios of catastrophic risk process large volumes of data. Often times these simulations are not performed in real-time scenarios as they are slow and consume large data. Such simulations can benefit from a framework that exploits parallelism for addressing the computational challenge and facilitates a distributed file system for addressing the data challenge. To this end, the Apache Hadoop framework is chosen for the simulation reported in this paper so that the computational challenge can be tackled using the MapReduce model and the data challenge can be addressed using the Hadoop Distributed File System. A parallel algorithm for the analysis of aggregate risk is proposed and implemented using the MapReduce model in this paper. An evaluation of the performance of the algorithm indicates that the Hadoop MapReduce model offers a framework for processing large data in aggregate risk analysis. A simulation of aggregate risk employing 100,000 trials with 1000 catastrophic events per trial on a typical exposure set and contract structure is performed on multiple worker nodes in less than 6 minutes. The result indicates the scope and feasibility of MapReduce for tackling the computational and data challenge in the analysis of aggregate risk for real-time use.
References
[1]
G. Connor, L. R. Goldberg and R. A. Korajczyk, "Portfolio Risk Analysis," Princeton University Press, 2010.
[2]
A. Melnikov, "Risk Analysis in Finance and Insurance," Second Edition, CRC Press, 2011.
[3]
A. K. Bahl, O. Baltzer, A. Rau-Chaplin and B. Varghese, "Parallel Simulations for Analysing Portfolios of Catastrophic Event Risk," Workshop of the International Conference for High Performance Computing, Networking, Storage and Analysis (SC), 2012.
[4]
J. Dean and S. Ghemawat, "MapReduce: Simplified Data Processing on Large Clusters," Communications of the ACM, Vol. 51, No. 1, 2008, pp. 107--113.
[5]
R. R. Anderson and W. Dong, "Pricing Catastrophe Reinsurance with Reinstatement Provisions Using a Catastrophe Model," Casualty Actuarial Society Forum, Summer 1998, pp. 303--322.
[6]
G. G. Meyers, F. L. Klinker and D. A. Lalonde, "The Aggregation and Correlation of Reinsurance Exposure," Casualty Actuarial Society Forum, Spring 2003, pp. 69--152.
[7]
W. Dong, H. Shah and F. Wong, "A Rational Approach to Pricing of Catastrophe Insurance," Journal of Risk and Uncertainty, Vol. 12, 1996, pp. 201--218.
[8]
K. -H. Lee, Y. -J. Lee, H. Choi, Y. D. Chung and B. Moon, "Parallel Data Processing with MapReduce: A Survey," SIGMOD Record, Vol. 40, No. 4, 2011, pp. 11--20.
[9]
T. Condie, N. Conway, P. Alvaro, J. M. Hellerstein, K. Elmeleegy and R. Sears, "MapReduce Online," EECS Department, University of California, Berkeley, USA, Oct 2009, Technical Report No. UCB/EECS-2009--136.
[10]
D. Cummins, C. Lewis and R. Phillips, "Pricing Excess-of-Loss Reinsurance Contracts Against Catastrophic Loss," The Financing of Catastrophe Risk, Editors: K. A. Froot, University of Chicago Press, 1999, pp. 93--148.
[11]
Y. -S. Lee, "The Mathematics of Excess of Loss Coverages and Retrospective Rating - A Graphical Approach," Casualty Actuarial Society Forum, 1988, pp. 49--77.
[12]
G. Woo, "Natural Catastrophe Probable Maximum Loss," British Actuarial Journal, Vol. 8, 2002.
[13]
M. E. Wilkinson, "Estimating Probable Maximum Loss with Order Statistics," Casualty Actuarial Society Forum, 1982, pp. 195--209.
[14]
A. A. Gaivoronski and G. Pflug, "Value-at-Risk in Portfolio Optimisation: Properties and Computational Approach," Journal of Risk, Vol. 7, No. 2, 2005, pp. 1--31.
[15]
P. Glasserman, P. Heidelberger and P. Shahabuddin, "Portfolio Value-at-Risk with Heavy-tailed Risk Factors," Mathematical Finance, Vol. 12, No. 3, 2002, pp. 239--269.
[16]
T. White, "Hadoop: The Definitive Guide," 1st Edition, O'Reilly Media, Inc., 2009.
[17]
Apache Hadoop Project: http://hadoop.apache.org/ {Last Accessed: 10 April, 2013}
[18]
K. Shvachko, K. Hairong, S. Radia and R. Chansler, "The Hadoop Distributed File System," Proceedings of the 26th IEEE Symposium on Mass Storage Systems and Technologies, 2010, pp. 1--10.
[19]
Amazon Elastic MapReduce (EMR): http://aws.amazon.com/elasticmapreduce/ {Last accessed: 10 April, 2013}
[20]
Google MapReduce: https://developers.google.com/appengine/docs/python/dataprocessing/overview {Last accessed: 10 April, 2013}
Index Terms
- High performance risk aggregation: addressing the data processing challenge the hadoop mapreduce way
Recommendations
A comparative between hadoop mapreduce and apache Spark on HDFS
IML '17: Proceedings of the 1st International Conference on Internet of Things and Machine LearningData is growing now in a very high speed with a large volume, Spark and MapReduce1 both provide a processing model for analyzing and managing this large data -Big Data- stored on HDFS. In this paper, we discuss a comparative between Apache Spark and ...
Overview of risk aggregation approach in different risk scenarios
AbstractRisk aggregation is the process of assessing the overall risk of multiple risks. Current research discusses risk aggregation issues in different risk scenarios, making the risk aggregation process not universal for diverse. But these works have ...
Query optimization using column statistics in hive
IDEAS '11: Proceedings of the 15th Symposium on International Database Engineering & ApplicationsHive is a data warehousing solution on top of the Hadoop MapReduce framework that has been designed to handle large amounts of data and store them in tables like a relational database management system or a conventional data warehouse while using the ...
Comments
Information & Contributors
Information
Published In
Copyright © 2013 ACM.
Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than ACM must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected]
Sponsors
- University of Arizona: University of Arizona
- SIGARCH: ACM Special Interest Group on Computer Architecture
Publisher
Association for Computing Machinery
New York, NY, United States
Publication History
Published: 17 June 2013
Check for updates
Author Tags
Qualifiers
- Research-article
Conference
HPDC'13
Sponsor:
- University of Arizona
- SIGARCH
HPDC'13: The 22nd International Symposium on High-Performance Parallel and Distributed Computing
June 17, 2013
New York, New York, USA
Acceptance Rates
Science Cloud '13 Paper Acceptance Rate 7 of 14 submissions, 50%;
Overall Acceptance Rate 44 of 151 submissions, 29%
Contributors
Other Metrics
Bibliometrics & Citations
Bibliometrics
Article Metrics
- 0Total Citations
- 183Total Downloads
- Downloads (Last 12 months)1
- Downloads (Last 6 weeks)1
Reflects downloads up to 28 Feb 2025
Other Metrics
Citations
View Options
Login options
Check if you have access through your login credentials or your institution to get full access on this article.
Sign in