High-performance composite event monitoring system supporting large numbers of queries and sources

ABSTRACT

This paper presents a novel data structure, called Event-centric Composable Queue (ECQ), a basic building block of a new scalable composite event monitoring (CEM) framework, SCEMon. In particular, we focus on the scalability issues when large numbers of CEM queries and event sources exist in upcoming CEM environments. To address these challenges effectively, we take an event-centric sharing approach rather than dealing with queries and sources separately. ECQ is a shared queue, which stores incoming event instances of a primitive event class. ECQs are designed to facilitate efficient shared evaluations of multiple queries over very large volumes of event streams from numerous event sources. ECQs are composable and form a single shared network within which multiple queries are simultaneously evaluated. In this paper, we present efficient shared processing techniques operating on top of the proposed shared ECQ network. The performance evaluation shows that the proposed approach achieves a high level of scalability compared to conventional separate processing approaches in large-scale CEM environments.

References

1. Wu, E., Diao, Y. and Rizvi, S. 2006. High-performance complex event processing over streams. In Proc. of SIGMOD.
2. Akdere, M., Çetintemel, U. and Tatbul, N. 2008. Plan-based Complex Event Detection across Distributed Sources. In Proc. of VLDB.
3. Mei, Y. and Madden, S. 2009. ZStream: A Cost-based Query Processor for Adaptively Detecting Composite Events. In Proc. of SIGMOD.
4. Yang, D., Rundensteiner, E. and Ward, M. 2009. A Shared Execution Strategy for Multiple Pattern Mining Requests over Streaming Data. In Proc. of VLDB.
5. Demers, A., Gehrke, J., Panda, B., Riedewald, M., Sharma, V. and White, W. 2007. Cayuga: A general purpose event monitoring system. In Proc. of CIDR.
6. Gatziu, S. and Dittrich, K. 1994. Events in an active object-oriented database, In Workshop on Rules in Database Systems.
7. Chakravarthy, S., Krishnaprasad, V., Anwar, E. and Kim, S. 1994. Composite events for active databases: Semantics, contexts and detection. In Proc. of VLDB.
8. Urban, S., Biswas, I. and Dietrich, S. 2006. Filtering features for a composite event definition language. In Proc. of SAINT.
9. Hinze, A. 2003. Efficient filtering of composite events, In Proc. of BNCD.
10. Elkhalifa, L., Adaikkalavan, R. and Chakravarthy, S. 2005. InfoFilter: A system for expressive pattern specification and detection over text streams. In Proc. of SAC.
11. Agrawal, J., Diao, Y., Gyllstrom, D. and Immerman, N. 2008 Efficient pattern matching over event streams. In Proc. of SIGMOD.
12. Ananthanarayanan, G., Haridasan, M., Mohomed, I., Terry, D. and Thekkath, C. 2009. StarTrack: A framework for enabling track-based applications. In Proc. of MobiSys.
13. Standard Template Library, http://www.cplusplus.com/reference/stl/
14. Cayuga source code, http://sourceforge.net/projects/cayuga/
15. Esper official site, http://esper.codehaus.org
16. Candan, K., Hsiung, W., Chen, S., Tatemura, J. and Agrawal, D. 2006. AFilter: Adaptable XML Filtering with Prefix-Caching and Suffix-Clustering. In Proc. of VLDB.
17. Hong, M., Riedewald, M., Koch, C., Gehrke, J. and Demers, A. 2009. Rule-based multi-query optimization. In Proc. of EDBT.

Index Terms

• Information systems

  • Data management systems

    • Database management system engines

      • Database query processing

  • Information retrieval

    • Retrieval tasks and goals

      • Document filtering

      • Information extraction

• Theory of computation

  • Theory and algorithms for application domains

    • Database theory

      • Database query processing and optimization (theory)