Abstract
Detecting the failure of a data stream is relatively easy when the stream is continually full of data. The transfer of large amounts of data allows for the simple detection of interference, whether accidental or malicious. However, during interference, data transmission can become irregular, rather than smooth. When the traffic is intermittent, it is harder to detect when failure has occurred and may lead to an application at the receiving end requesting retransmission or disconnecting. Request retransmission places additional load on a system and disconnection can lead to unnecessary reversion to a checkpointed database, before reconnecting and reissuing the same request or response. In this paper, we model the traffic in data streams as a set of significant events, with an arrival rate distributed with a Poisson distribution. Once an arrival rate has been determined, over-time, or lost, events can be determined with a greater chance of reliability. This model also allows for the alteration of the rate parameter to reflect changes in the system and provides support for multiple levels of data aggregation. One significant benefit of the Poisson-based model is that transmission events can be deliberately manipulated in time to provide a steganographic channel that confirms sender/receiver identity.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Preview
Unable to display preview. Download preview PDF.
References
Golab, L., Özsu, M.T.: Issues in data stream management. SIGMOD Rec. 32(2), 5–14 (2003)
Gaber, M.M., Zaslavsky, A., Krishnaswamy, S.: Mining Data Streams: A Review. SIGMOD Rec. 34(2), 18–26 (2005)
Babcock, B., Babu, S., Datar, M., Motwani, R., Widom, J.: Models and Issues in Data Stream Systems. In: PODS 2002: Proc. of the 21st ACM SIGMOD-SIGACT-SIGART Symposium on Principles of Database Systems, pp. 1–16. ACM, New York (2002)
Babu, S., Widom, J.: Continuous Queries over Data Streams. SIGMOD Rec. 30(3), 109–120 (2001)
Berndt, D.J., Clifford, J.: Using Dynamic Time Warping to Find Patterns in Time Series. In: AAAI 1994 Workshop on Knowledge Discovery in Databases, pp. 359–370. AAAI Press, Menlo Park (1994)
Papadimitriou, S., Sun, J., Faloutsos, C.: Streaming Pattern Discovery in Multiple Time-series. In: VLDB 2005: Proc. of the 31st Intl. Conference on Very Large Data Bases, pp. 697–708. ACM, New York (2005)
Bai, Y., Wang, F., Liu, P.: Efficiently filtering RFID data streams. In: CleanDB: The First International VLDB Workshop on Clean Databases, pp. 50–57. ACM, New York (2006)
Wei, Y., Son, S.H., Stankovic, J.A.: RTSTREAM: Real-Time Query Processing for Data Streams. In: 9th IEEE International Symposium on Object/component/service-oriented Real-Time Distributed Computing, pp. 141–150 (2006)
Zhu, Y., Shasha, D.: StatStream: Statistical Monitoring of Thousands of Data Streams in Real Time. In: VLDB 2002: Proc. of the 28th Intl. Conference on Very Large Data Bases, VLDB Endowment, pp. 358–369 (2002)
Gu, L., Jia, D., Vicaire, P., Yan, T., Luo, L., Tirumala, A., Cao, Q., He, T., Stankovic, J.A., Abdelzaher, T., Krogh, B.H.: Lightweight Detection and Classification for Wireless Sensor Networks in Realistic Environments. In: SenSys 2005: Proc. of the 3rd Intl. Conference on Embedded Networked Sensor Systems, pp. 205–217. ACM, New York (2005)
Solis, I., Obraczka, K.: In-Network Aggregation Trade-offs for Data Collection in Wireless Sensor Networks. Intl. Journal of Sensor Networks 1(3–4), 200–212 (2007)
Ye, F., Luo, H., Lu, S., Zhang, L.: Statistical En-Route Filtering of Injected False Data in Sensor Networks. IEEE Journal on Selected Areas in Communications 23(4), 839–850 (2005)
Pottie, G.J., Kaiser, W.J.: Wireless Integrated Network Sensors. Commun. ACM 43(5), 51–58 (2000)
Feng, J., Koushanfar, F., Potkonjak, M.: Sensor Network Architecture. Number 12 in III. In: Handbook of Sensor Networks. CRC Press, Boca Raton (2004)
Madden, S., Franklin, M.J., Hellerstein, J.M., Hong, W.: TAG: a Tiny AGgregation Service for Ad-hoc Sensor Networks. SIGOPS Oper. Syst. Rev. 36(SI), 131–146 (2002)
Petrovic, M., Burcea, I., Jacobsen, H.A.: S-ToPSS: Semantic Toronto Publish/Subscribe System. In: VLDB 2003: Proc. of the 29th Intl. Conference on Very Large Data Bases, VLDB Endowment, pp. 1101–1104 (2003)
Gupta, P., Kumar, P.R.: The Capacity of Wireless Sensor Networks. IEEE Trans. Info. Theory 46(2) (2000)
Levis, P., Lee, N., Welsh, M., Culler, D.: TOSSIM: Accurate and Scalable Simulation of Entire TinyOS Applications. In: SenSys ’03: Proc. of the 1st Intl. Conference on Embedded Networked Sensor Systems, pp. 126–137. ACM, New York (2003)
Author information
Authors and Affiliations
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2009 Springer-Verlag Berlin Heidelberg
About this paper
Cite this paper
Falkner, N.J.G., Sheng, Q.Z. (2009). Significance-Based Failure and Interference Detection in Data Streams. In: Bhowmick, S.S., Küng, J., Wagner, R. (eds) Database and Expert Systems Applications. DEXA 2009. Lecture Notes in Computer Science, vol 5690. Springer, Berlin, Heidelberg. https://doi.org/10.1007/978-3-642-03573-9_54
Download citation
DOI: https://doi.org/10.1007/978-3-642-03573-9_54
Publisher Name: Springer, Berlin, Heidelberg
Print ISBN: 978-3-642-03572-2
Online ISBN: 978-3-642-03573-9
eBook Packages: Computer ScienceComputer Science (R0)