Abstract
Triangle enumeration is a fundamental task in graph data analysis with many applications. Recently, Park et al. proposed a distributed algorithm, PTE (Pre-partitioned Triangle Enumeration), that, unlike previous works, scales well using multiple high end machines and can handle very large real-world networks.
This work presents a serverless implementation of the PTE algorithm using the AWS Lambda platform. Our experiments take advantage of the high concurrency of the lambda instances to compete with the expensive server-based experiments of Park et al. Our analysis shows the trade-off between the time and cost of triangle enumeration and the numbers of tasks generated by the distributed algorithm. Our results reveal the importance of using a higher number of tasks in order to improve the efficiency of PTE. Such an analysis can only be performed using a large number of workers which is indeed possible using AWS Lambda but not easy to achieve using few servers as in the case of Park et al.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
References
Dementiev, R.: Algorithm engineering for large data sets, Ph.D. dissertation, Verlag nicht ermittelbar (2006)
Menegola, B.: An external memory algorithm for listing triangles (2010)
Hu, X., Tao, Y., Chung, C.-W.: Massive graph triangulation. In: Proceedings of the 2013 ACM SIGMOD International Conference on Management of Data, pp. 325–336 (2013)
Park, H.-M., Myaeng, S.-H., Kang, U.: PTE: enumerating trillion triangles on distributed systems. In: Proceedings of the 22nd ACM SIGKDD International Conference on Knowledge Discovery and Data Mining, pp. 1115–1124 (2016)
Arifuzzaman, S., Khan, M., Marathe, M.: PATRIC: a parallel algorithm for counting triangles in massive networks. In: Proceedings of the 22nd ACM International Conference on Information & Knowledge Management, pp. 529–538 (2013)
Giechaskiel, I., Panagopoulos, G., Yoneki, E.: PDTL: parallel and distributed triangle listing for massive graphs. In: 2015 44th International Conference on Parallel Processing, pp. 370–379. IEEE (2015)
Cohen, J.: Graph twiddling in a mapreduce world. Comput. Sci. Eng. 11(4), 29–41 (2009)
Suri, S., Vassilvitskii, S.: Counting triangles and the curse of the last reducer. In: Proceedings of the 20th International Conference on World Wide Web, pp. 607–614 (2011)
Park, H.-M., Chung, C.-W.: An efficient mapreduce algorithm for counting triangles in a very large graph. In: Proceedings of the 22nd ACM International Conference on Information & Knowledge Management, pp. 539–548 (2013)
Park, H.-M., Silvestri, F., Kang, U., Pagh, R.: Mapreduce triangle enumeration with guarantees. In: Proceedings of the 23rd ACM International Conference on Conference on Information and Knowledge Management, pp. 1739–1748 (2014)
Wikipedia contributors: AWS lambda—Wikipedia, the free encyclopedia (2020). https://en.wikipedia.org/w/index.php?title=AWS_Lambda. Accessed 10 Apr 2020
Amazon Web Service: Configuring functions in the AWS lambda console (2020). https://docs.aws.amazon.com/lambda/latest/dg/configuration-console.html
Amazon Web Service: Amazon EC2 pricing (2020). https://aws.amazon.com/ec2/pricing/on-demand/
Boldi, P., Vigna, S.: The WebGraph framework I: compression techniques. In: Proceedings of the Thirteenth International World Wide Web Conference (WWW 2004), Manhattan, USA, pp. 595–601. ACM Press (2004)
Boldi, P., Rosa, M., Santini, M., Vigna, S.: Layered label propagation: a multiresolution coordinate-free ordering for compressing social networks. In: Srinivasan, S., Ramamritham, K., Kumar, A., Ravindra, M.P., Bertino, E., Kumar, R. (eds.) Proceedings of the 20th International Conference on World Wide Web, pp. 587–596. ACM Press (2011)
Chen, S., Wei, R., Popova, D., Thomo, A.: Efficient computation of importance based communities in web-scale networks using a single machine. In: Proceedings of the 25th ACM International on Conference on Information and Knowledge Management, pp. 1553–1562. ACM (2016)
Esfahani, F., Srinivasan, V., Thomo, A., Wu, K.: Efficient computation of probabilistic core decomposition at web-scale. In: Advances in Database Technology-EDBT 2019, 22nd International Conference on Extending Database Technology, pp. 325–336 (2019)
Khaouid, W., Barsky, M., Srinivasan, V., Thomo, A.: K-core decomposition of large networks on a single PC. Proc. VLDB Endow. 9(1), 13–23 (2015)
Popova, D., Ohsaka, N., Kawarabayashi, K., Thomo, A.: NoSingles: a space-efficient algorithm for influence maximization. In: Proceedings of the 30th International Conference on Scientific and Statistical Database Management, p. 18. ACM (2018)
Simpson, M., Srinivasan, V., Thomo, A.: Clearing contamination in large networks. IEEE Trans. Knowl. Data Eng. 28(6), 1435–1448 (2016)
Simpson, M., Srinivasan, V., Thomo, A.: Efficient computation of feedback arc set at web-scale. Proc. VLDB Endow. 10(3), 133–144 (2016)
Santoso, Y., Thomo, A., Srinivasan, V., Chester, S.: Triad enumeration at trillion-scale using a single commodity machine. In: Advances in Database Technology-EDBT 2019, 22nd International Conference on Extending Database Technology. OpenProceedings.org (2019)
Santoso, Y., Srinivasan, V., Thomo, A.: Efficient enumeration of four node graphlets at trillion-scale. In: Advances in Database Technology-EDBT 2020, 23rd International Conference on Extending Database Technology, pp. 439–442 (2020)
Esfahani, F., Wu, J., Srinivasan, V., Thomo, A., Wu, K.: Fast truss decomposition in large-scale probabilistic graphs. In: Advances in Database Technology-EDBT 2019, 22nd International Conference on Extending Database Technology, pp. 722–725 (2019)
Wu, J., Goshulak, A., Srinivasan, V., Thomo, A.: K-truss decomposition of large networks on a single consumer-grade machine. In: 2018 IEEE/ACM International Conference on Advances in Social Networks Analysis and Mining (ASONAM), pp. 873–880. IEEE (2018)
Author information
Authors and Affiliations
Corresponding authors
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2021 Springer Nature Switzerland AG
About this paper
Cite this paper
Yu, T., Srinivasan, V., Thomo, A. (2021). Triangle Enumeration on Massive Graphs Using AWS Lambda Functions. In: Barolli, L., Li, K., Miwa, H. (eds) Advances in Intelligent Networking and Collaborative Systems. INCoS 2020. Advances in Intelligent Systems and Computing, vol 1263. Springer, Cham. https://doi.org/10.1007/978-3-030-57796-4_22
Download citation
DOI: https://doi.org/10.1007/978-3-030-57796-4_22
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-030-57795-7
Online ISBN: 978-3-030-57796-4
eBook Packages: Intelligent Technologies and RoboticsIntelligent Technologies and Robotics (R0)