skip to main content
10.1145/3524860.3539652acmconferencesArticle/Chapter ViewAbstractPublication PagesdebsConference Proceedingsconference-collections
short-paper

Real-time stock market analytics for improving deployment and accessibility using PySpark and Docker

Published:15 July 2022Publication History

ABSTRACT

Making timely-decisions amid the massive influx of financial data is one of the essential features of stock market analytics. Many stock market analytics should provide functionalities that compute multiple technical indicators simultaneously and detect breakout situations. The DEBS 2022 Grand Challenge (DEBS22 GC) competition requires to answering two types of queries: technical trend indicators and detection of crossover patterns. In response to the competition, we propose a real-time stock market analytic solution using PySpark and Docker. Our solution calculates the technical trend indicator---Exponential Moving Average(EMA)---in real-time with the window function. With the technical indicators computed, we detect the breakout pattern that helps determine either buy or sell stocks. Our solution not only improves the speed of deploying applications using a Docker container image but also can be accessed easily via a web-based Jupyter notebook.

References

  1. Sebastian Frischbier, Jawad Tahir, Christoph Doblander, Arne Hormann, Ruben Mayer, and Hans-Arno Jacobsen. 2022. The DEBS 2022 Grand Challenge: Detecting Trading Trends in Financial Tick Data. In The 16th ACM International Conference on Distributed and Event-based Systems (DEBS '22), June 27-June 30, 2022, Copenhagen. ACM, New York, NY, USA. Google ScholarGoogle ScholarDigital LibraryDigital Library
  2. Chris Bradley and Peter Stumpner. 2021. The impact of COVID-19 on capital markets, one year in. McKinsey.Google ScholarGoogle Scholar
  3. Sebastian Frischbier, Jawad Tahir, Christoph Doblander, Arne Hormann, Ruben Mayer, and Hans-Arno Jacobsen. 2022. DEBS 2022 Grand Challenge Data Set: Trading Data. Google ScholarGoogle ScholarCross RefCross Ref
  4. Raul Castro Fernandez, Matthias Weidlich, Peter Pietzuch, and Avigdor Gal. 2014. Scalable stateful stream processing for smart grids. In Proceedings of the 8th ACM International Conference on Distributed Event-Based Systems, 276--281.Google ScholarGoogle ScholarDigital LibraryDigital Library
  5. Vadlamani Ravi, Dadabada Pradeepkumar, and Kalyanmoy Deb. 2017. Financial time series prediction using hybrids of chaos theory, multi-layer perceptron and multi-objective evolutionary algorithms. Swarm and Evolutionary Computation 36, 136--149.Google ScholarGoogle ScholarCross RefCross Ref
  6. Jules S. Damji, Brooke Wenig, Tathagata Das, and Denny Lee. 2020. Learning Spark. O'Reilly Media.Google ScholarGoogle Scholar
  7. Anand Gupta, Hardeo Kumar Thakur, Ritvik Shrivastava, Pulkit Kumar, and Sreyashi Nag. 2017. A big data analysis framework using apache spark and deep learning. In 2017 IEEE international conference on data mining workshops (ICDMW), 9--16.Google ScholarGoogle ScholarCross RefCross Ref
  8. Pedro Silva, Wang Yue, and Tilmann Rabl. Incremental stream query analytics. 2020. In Proceedings of the 14th ACM International Conference on Distributed and Event-Based Systems(DEBS), 187--192.Google ScholarGoogle Scholar
  9. Vincenzo Gulisano, Daniel Jorde, Ruben Mayer, Hannaneh Najdataei, and Dimitris Palyvos-Giannas. 2020. The DEBS 2020 grand challenge. In Proceedings of the 14th ACM International Conference on Distributed and Event-Based Systems, 183--186.Google ScholarGoogle Scholar
  10. Sejin Chun, Seungjun Yoon, Jooik Jung, and Kyong-Ho Lee, 2019. Planning Operators of Concurrent RDF Stream Processing Queries. International Journal of Web and Grid Services (IJWGS), Volume 15. No. 1.Google ScholarGoogle ScholarCross RefCross Ref

Index Terms

  1. Real-time stock market analytics for improving deployment and accessibility using PySpark and Docker

      Recommendations

      Comments

      Login options

      Check if you have access through your login credentials or your institution to get full access on this article.

      Sign in
      • Published in

        cover image ACM Conferences
        DEBS '22: Proceedings of the 16th ACM International Conference on Distributed and Event-Based Systems
        June 2022
        210 pages
        ISBN:9781450393089
        DOI:10.1145/3524860

        Copyright © 2022 ACM

        Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than ACM must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected]

        Publisher

        Association for Computing Machinery

        New York, NY, United States

        Publication History

        • Published: 15 July 2022

        Permissions

        Request permissions about this article.

        Request Permissions

        Check for updates

        Qualifiers

        • short-paper

        Acceptance Rates

        DEBS '22 Paper Acceptance Rate10of19submissions,53%Overall Acceptance Rate130of553submissions,24%

        Upcoming Conference

        DEBS '24
      • Article Metrics

        • Downloads (Last 12 months)49
        • Downloads (Last 6 weeks)7

        Other Metrics

      PDF Format

      View or Download as a PDF file.

      PDF

      eReader

      View online with eReader.

      eReader