Skip to main content
SpringerLink
Log in
Book cover

International Symposium on Automated Technology for Verification and Analysis

ATVA 2019: Automated Technology for Verification and Analysis pp 133–150Cite as

Adaptive Online First-Order Monitoring

  • Conference paper
  • First Online:

Part of the book series: Lecture Notes in Computer Science ((LNPSE,volume 11781))

Abstract

Online first-order monitoring is the task of detecting temporal patterns in streams of events carrying data. Considerable research has been devoted to scaling up monitoring using parallelization by partitioning events based on their data values and processing the partitions concurrently. To be effective, partitioning must account for the event stream’s statistics, e.g., the relative event frequencies, and these statistics may change rapidly. We develop the first parallel online first-order monitor capable of adapting to such changes. A central problem we solve is how to manage and exchange states between the parallel executing monitors. To this end, we develop state exchange operations and prove their correctness. Moreover, we extend the implementation of the MonPoly monitoring tool with these operations, thereby supporting parallel adaptive monitoring, and show empirically that adaptation can yield up to a tenfold improvement in run-time.

This is a preview of subscription content, log in via an institution.

Chapter
USD   29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
eBook
USD   39.99
Price excludes VAT (USA)
  • Available as EPUB and PDF
  • Read on any device
  • Instant download
  • Own it forever
Softcover Book
USD   54.99
Price excludes VAT (USA)
  • Compact, lightweight edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info

Tax calculation will be finalised at checkout

Purchases are for personal use only

Learn about institutional subscriptions

References

  1. Afrati, F.N., Ullman, J.D.: Optimizing multiway joins in a map-reduce environment. IEEE Trans. Knowl. Data Eng. 23(9), 1282–1298 (2011)

    Article  Google Scholar 

  2. Alexandrov, A., et al.: The Stratosphere platform for big data analytics. VLDB J. 23(6), 939–964 (2014)

    Article  Google Scholar 

  3. Alur, R., Mamouras, K., Stanford, C.: Modular quantitative monitoring. PACMPL 3(POPL), 50:1–50:31 (2019)

    Google Scholar 

  4. Aniello, L., Baldoni, R., Querzoni, L.: Adaptive online scheduling in Storm. In: DEBS 2013, pp. 207–218. ACM (2013)

    Google Scholar 

  5. Barre, B., Klein, M., Soucy-Boivin, M., Ollivier, P.-A., Hallé, S.: MapReduce for parallel trace validation of LTL properties. In: Qadeer, S., Tasiran, S. (eds.) RV 2012. LNCS, vol. 7687, pp. 184–198. Springer, Heidelberg (2013). https://doi.org/10.1007/978-3-642-35632-2_20

    Chapter  Google Scholar 

  6. Basin, D., Caronni, G., Ereth, S., Harvan, M., Klaedtke, F., Mantel, H.: Scalable offline monitoring of temporal specifications. Form. Methods Syst. Des. 49(1–2), 75–108 (2016)

    Article  Google Scholar 

  7. Basin, D., Klaedtke, F., Müller, S., Zălinescu, E.: Monitoring metric first-order temporal properties. J. ACM 62(2), 15:1–15:45 (2015)

    Article  MathSciNet  Google Scholar 

  8. Basin, D., Klaedtke, F., Zălinescu, E.: The MonPoly monitoring tool. In: Reger, G., Havelund, K. (eds.) RV-CuBES 2017. Kalpa Publications in Computing, vol. 3, pp. 19–28. EasyChair (2017)

    Google Scholar 

  9. Basin, D., Klaedtke, F., Zălinescu, E.: Greedily computing associative aggregations on sliding windows. Inf. Process. Lett. 115(2), 186–192 (2015)

    Article  MathSciNet  Google Scholar 

  10. Beame, P., Koutris, P., Suciu, D.: Communication steps for parallel query processing. J. ACM 64(6), 40:1–40:58 (2017)

    Article  MathSciNet  Google Scholar 

  11. Bersani, M.M., Bianculli, D., Ghezzi, C., Krstić, S., Pietro, P.S.: Efficient large-scale trace checking using MapReduce. In: Dillon, L.K., Visser, W., Williams, L. (eds.) ICSE 2016. pp. 888–898. ACM (2016)

    Google Scholar 

  12. Bianculli, D., Ghezzi, C., Krstić, S.: Trace checking of metric temporal logic with aggregating modalities using MapReduce. In: Giannakopoulou, D., Salaün, G. (eds.) SEFM 2014. LNCS, vol. 8702, pp. 144–158. Springer, Cham (2014). https://doi.org/10.1007/978-3-319-10431-7_11

    Chapter  Google Scholar 

  13. Cormode, G., Garofalakis, M.N., Haas, P.J., Jermaine, C.: Synopses for massive data: samples, histograms, wavelets, sketches. Found. Trends Databases 4(1–3), 1–294 (2012)

    MATH  Google Scholar 

  14. Elseidy, M., Elguindy, A., Vitorovic, A., Koch, C.: Scalable and adaptive online joins. PVLDB 7(6), 441–452 (2014)

    Google Scholar 

  15. Hallé, S., Soucy-Boivin, M.: MapReduce for parallel trace validation of LTL properties. J. Cloud Comput. 4(1), 8 (2015)

    Article  Google Scholar 

  16. Hoffmann, M., Lattuada, A., McSherry, F., Kalavri, V., Liagouris, J., Roscoe, T.: Megaphone: latency-conscious state migration for distributed streaming dataflows. PVLDB 12(9), 1002–1015 (2019)

    Google Scholar 

  17. Kalavri, V., Liagouris, J., Hoffmann, M., Dimitrova, D.C., Forshaw, M., Roscoe, T.: Three steps is all you need: fast, accurate, automatic scaling decisions for distributed streaming dataflows. In: Arpaci-Dusseau, A.C., Voelker, G. (eds.) OSDI 2018, pp. 783–798. USENIX Association (2018)

    Google Scholar 

  18. Kolchinsky, I., Schuster, A.: Efficient adaptive detection of complex event patterns. PVLDB 11(11), 1346–1359 (2018)

    Google Scholar 

  19. Mamouras, K., Raghothaman, M., Alur, R., Ives, Z.G., Khanna, S.: StreamQRE: modular specification and efficient evaluation of quantitative queries over streaming data. In: Cohen, A., Vechev, M.T. (eds.) PLDI 2017, pp. 693–708. ACM (2017)

    Article  Google Scholar 

  20. Mayer, R., Tariq, M.A., Rothermel, K.: Minimizing communication overhead in window-based parallel complex event processing. In: DEBS 2017, pp. 54–65. ACM (2017)

    Google Scholar 

  21. Murray, D.G., McSherry, F., Isaacs, R., Isard, M., Barham, P., Abadi, M.: Naiad: a timely dataflow system. In: Kaminsky, M., Dahlin, M. (eds.) SOSP 2013, pp. 439–455. ACM (2013)

    Google Scholar 

  22. Reger, G., Rydeheard, D.: From first-order temporal logic to parametric trace slicing. In: Bartocci, E., Majumdar, R. (eds.) RV 2015. LNCS, vol. 9333, pp. 216–232. Springer, Cham (2015). https://doi.org/10.1007/978-3-319-23820-3_14

    Chapter  Google Scholar 

  23. Roşu, G., Chen, F.: Semantics and algorithms for parametric monitoring. Log. Methods Comput. Sci. 8(1:9), 1–47 (2012)

    Google Scholar 

  24. Schneider, J., Basin, D., Brix, F., Krstić, S., Traytel, D.: Scalable online first-order monitoring. In: Colombo, C., Leucker, M. (eds.) RV 2018. LNCS, vol. 11237, pp. 353–371. Springer, Cham (2018). https://doi.org/10.1007/978-3-030-03769-7_20

    Chapter  Google Scholar 

  25. Schneider, J., Basin, D., Brix, F., Krstić, S., Traytel, D.: Artifact associated with this paper (2019). https://bitbucket.org/jshs/monpoly/downloads/aom_atva2019.zip

  26. Schneider, J., Basin, D., Krstić, S., Traytel, D.: A formally verified monitor for metric first-order temporal logic. In: Finkbeiner, B., Mariani, L. (eds.) RV 2019. LNCS, Springer (2019, to appear). http://people.inf.ethz.ch/trayteld/papers/rv19-verimon/verimon.pdf

  27. Shah, M.A., Hellerstein, J.M., Chandrasekaran, S., Franklin, M.J.: Flux: an adaptive partitioning operator for continuous query systems. In: Dayal, U., Ramamritham, K., Vijayaraman, T.M. (eds.) ICDE 2003, pp. 25–36. IEEE (2003)

    Google Scholar 

  28. Vitorovic, A., et al.: Squall: Scalable real-time analytics. PVLDB 9(13), 1553–1556 (2016)

    Google Scholar 

Download references

Acknowledgment

Christian Fania helped us implement and evaluate our adaptive monitoring framework. The anonymous reviewers gave numerous helpful comments on earlier drafts of this paper. Joshua Schneider is supported by the US Air Force grant “Monitoring at Any Cost” (FA9550-17-1-0306). Srđan Krstić is supported by the Swiss National Science Foundation grant “Big Data Monitoring” (167162).

Author information

Authors and Affiliations

  1. Institute of Information Security, Department of Computer Science, ETH Zürich, Zurich, Switzerland

    Joshua Schneider, David Basin, Frederik Brix, Srđan Krstić & Dmitriy Traytel

Authors
  1. Joshua Schneider
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. David Basin
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Frederik Brix
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Srđan Krstić
    View author publications

    You can also search for this author in PubMed Google Scholar

  5. Dmitriy Traytel
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding authors

Correspondence to Joshua Schneider , Srđan Krstić or Dmitriy Traytel .

Editor information

Editors and Affiliations

  1. Institute of Information Science, Academia Sinica, Taipei, Taiwan

    Yu-Fang Chen

  2. DENSO AUTOMOTIVE Deutschland GmbH, Eching, Germany

    Chih-Hong Cheng

  3. Institute of Computer Science, TU München, Munich, Germany

    Javier Esparza

Rights and permissions

Reprints and permissions

Copyright information

© 2019 Springer Nature Switzerland AG

About this paper

Check for updates. Verify currency and authenticity via CrossMark

Cite this paper

Schneider, J., Basin, D., Brix, F., Krstić, S., Traytel, D. (2019). Adaptive Online First-Order Monitoring. In: Chen, YF., Cheng, CH., Esparza, J. (eds) Automated Technology for Verification and Analysis. ATVA 2019. Lecture Notes in Computer Science(), vol 11781. Springer, Cham. https://doi.org/10.1007/978-3-030-31784-3_8

Download citation

  • DOI: https://doi.org/10.1007/978-3-030-31784-3_8

  • Published:

  • Publisher Name: Springer, Cham

  • Print ISBN: 978-3-030-31783-6

  • Online ISBN: 978-3-030-31784-3

  • eBook Packages: Computer ScienceComputer Science (R0)

Publish with us

Policies and ethics

Search

Navigation