SMILE: distributed middleware for event stream processing

In this paper, we describe the SMILE (Smart MIddleware, Light Ends) system which is one of the earliest systems built in the area of distributed event stream processing. SMILE unites the "publish-subscribe" model of messaging middleware with the "continuous query" model of database systems. In SMILE, information producers, which may be sensors, applications, or databases, generate streams of events, such as RFID data, news items or stock trades; consumers specify stateful subscriptions to derived views, such as "individuals trading top 5 total volume of stock Y within x minutes before a major news story about the same company Y"; the SMILE system constructs and deploys a dataflow network of computations which process events from producers, compute derived views and deliver continuous and timely updates of subscribed views to consumers. Research challenges addressed by SMILE include: rigorously defining the semantics for correct state delivery; implementing this semantics in the presence of failure; distributing computations optimally over a network of multiple servers; scheduling and controlling message flow in this network; smoothly integrating this technology with other components, e.g., databases, services, user interfaces. SMILE is applicable to any "sense and respond" scenario that requires monitoring and processing of events as they are generated. Applications span multiple industries, e.g., monitoring financial opportunities and detecting potential fraud in financial industry, systems management and alerts in data centers, inventory management in RFID applications, and business performance management. In this paper, we describe the SMILE system, its architecture, services supported, and implementation details, and its use in multiple application scenarios.