Abstract
With increasing popularity of cloud services, the microservices architecture has been gaining more attention in the software development industry. The idea of the microservices architecture is to use a collection of loosely coupled services to compose a large-scale software application. In traditional monolithic architecture, by contrast, every piece of code is put together, and the application is developed, tested, and deployed as a single application. Obviously, it is challenging for the traditional architecture to scale properly. In this research, we implemented a social data analytics platform based on the microservices architecture over DC/OS. Specifically, our data analytics service is built by composing many open-source software including Spark, Kafka, and Node.js. On streaming processing, our platform offers a visual interface to show the hottest hashtags of the most popular user posts from an online forum. On batch processing, our platform is able to show the statistics such the top-10 liked or commented posts and the gender counts of the posters. The experimental results show that our data analytics platform can do streaming processing and batch processing successfully and reveal useful analytical results.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
References
Hu, H., Wen, Y., Gao, Y., Chua, T.S., Li, X.: Toward an SDN-enabled big data platform for social TV analytics. IEEE Netw. 29(5), 43–49 (2015). https://doi.org/10.1109/MNET.2015.7293304
Enshaeifar, S., Barnaghi, P., Skillman, S., Markides, A., Elsaleh, T., Acton, S.T., Nilforooshan, R., Rostill, H.: The Internet of Things for dementia care. IEEE Internet Comput. 22(1), 8–17 (2018)
Swedberg, C.: Japanese Hospital tests BLE beacons to track patient–staff interactions. RFID J. (2018). http://www.rfidjournal.com/articles/view?17294. Accessed 25 April 2018
Newman, S.: Building Microservices. O’Reilly Media, Inc, Newton (2015)
Thönes, J.: Microservices. IEEE Softw. 32(1), 116 (2015)
Fielding, R.T.: Architectural styles and the design of network-based software architectures. University of California, Irvine (2000)
The definitive platform for modern apps - DC/OS. https://dcos.io/. Accessed 4 May 2018
Apache Spark. https://spark.apache.org/. Accessed 4 May 2018
Apache Kafka. https://kafka.apache.org/. Accessed 4 May 2018
Apache Mesos. http://mesos.apache.org/. Accessed 4 May 2018
Docker. https://www.docker.com/. Accessed 4 May 2018
Apache Hadoop. http://hadoop.apache.org/. Accessed 4 May 2018
Apache Cassandra. http://cassandra.apache.org/. Accessed 4 May 2018
Marathon. https://mesosphere.github.io/marathon/. Accessed 4 May 2018
Apache ZooKeeper. https://zookeeper.apache.org/. Accessed 4 May 2018
Linux Containers. https://linuxcontainers.org/. Accessed 4 May 2018
Kubernetes. https://kubernetes.io/. Accessed 6 May 2018
Eder, M.: Hypervisor- vs. container-based virtualization. In: Seminars Future Internet (FI) and Innovative Internet Technologies and Mobile Communications (IITM), Winter Semester 2015/2016, Munich, Germany, July 2016, pp. 1–7 (2016)
Zaharia, M., Chowdhury, M., Das, T., Dave, A., Ma, J., McCauley, M., Franklin, M.J., Shenker, S., Stoica, I.: Resilient distributed datasets: a fault-tolerant abstraction for in-memory cluster computing. In: 9th USENIX Conference on Networked Systems Design and Implementation, San Jose, CA, 25–27 April 2012 (2012)
Fazio, M., Celesti, A., Ranjan, R., Liu, C., Chen, L., Villari, M.: Open issues in scheduling microservices in the cloud. IEEE Cloud Comput. 3(5), 81–88 (2016)
Hill, R., Shadija, D., Rezai, M.: Enabling community healthcare with microservices. Paper presented at the 16th IEEE international conference on ubiquitous computing and communications, Guangzhou, China, 12–15 December 2017 (2017)
Le, V.D., Neff, M.M., Stewart, R.V., Kelley, R., Fritzinger, E., Dascalu, S.M., Harris, F.C.: Microservice-based architecture for the NRDC. In: 2015 IEEE 13th International Conference on Industrial Informatics (INDIN), 22–24 July 2015, pp. 1659–1664
Song, Y., Alatorre, G., Mandagere, N., Singh, A.: Storage mining: where IT management meets big data analytics. In: 2013 IEEE International Congress on Big Data, June 27–July 2 2013. pp. 421–422
Shyam, R., Ganesh, H.B.B., Kumar, S.S., Poornachandran, P., Soman, K.P.: Apache Spark a big data analytics platform for smart grid. Procedia Technol. 21, 171–178 (2015)
Nastic, S., Rausch, T., Scekic, O., Dustdar, S., Gusev, M., Koteska, B., Kostoska, M., Jakimovski, B., Ristov, S., Prodan, R.: A serverless real-time data analytics platform for edge computing. IEEE Internet Comput. 21(4), 64–71 (2017)
Lee, C.H., Lin, C.Y. Implementation of Lambda architecture: a restaurant recommender system over apache Mesos. In: 2017 IEEE 31st International Conference on Advanced Information Networking and Applications (AINA), 27–29 March 2017, pp. 979–985
Apache Kafka 0.9 client for Node. https://github.com/oleksiyk/kafka. Accessed 8 May 2018
Routing. https://expressjs.com/en/guide/routing.html. Accessed 25 May 2018
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2019 Springer Nature Switzerland AG
About this paper
Cite this paper
Hsu, MC., Lin, CY. (2019). A Microservices-Based Social Data Analytics Platform Over DC/OS. In: Barolli, L., Kryvinska, N., Enokido, T., Takizawa, M. (eds) Advances in Network-Based Information Systems. NBiS 2018. Lecture Notes on Data Engineering and Communications Technologies, vol 22. Springer, Cham. https://doi.org/10.1007/978-3-319-98530-5_58
Download citation
DOI: https://doi.org/10.1007/978-3-319-98530-5_58
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-319-98529-9
Online ISBN: 978-3-319-98530-5
eBook Packages: Intelligent Technologies and RoboticsIntelligent Technologies and Robotics (R0)