Challenge: On Online Time Series Clustering for Demand Response: Optic - A Theory to Break the 'Curse of Dimensionality'

The advent of smart meters and advanced communication infrastructures catalyzes numerous smart grid applications such as dynamic demand response, and paves the way to solve challenging research problems in sustainable energy consumption. The space of solution possibilities are restricted primarily by the huge amount of generated data requiring considerable computational resources and efficient algorithms. To overcome this Big Data challenge, data clustering techniques have been proposed. Current approaches however do not scale in the face of the "increasing dimensionality" problem, where a cluster point is represented by the entire customer consumption time series. To overcome this aspect we first rethink the way cluster points are created and designed, and then devise OPTIC, an efficient online time series clustering technique for demand response (DR), in order to analyze high volume, high dimensional energy consumption time series data at scale, and on the fly. OPTIC is randomized in nature, and provides optimal performance guarantees (Section 2.3.2) in a computationally efficient manner. Unlike prior work we (i) study the consumption properties of the whole population simultaneously rather than developing individual models for each customer separately, claiming it to be a 'killer' approach that breaks the "of dimensionality" in online time series clustering, and (ii) provide tight performance guarantees in theory to validate our approach. Our insights are driven by the field of sociology, where collective behavior often emerges as the result of individual patterns and lifestyles. We demonstrate the efficacy of OPTIC in practice using real-world data obtained from the fully operational USC microgrid.