Outlier detection is an important task in data mining, with applications ranging from intrusion detection to human gait analysis. With the growing need to analyze high speed data streams, the task of outlier detection becomes even more challenging as traditional outlier detection techniques can no longer assume that all the data can be stored for processing. While researchers mostly focus on detecting global outliers for data streams, detecting local outliers on streaming data has been neglected. This is an example of the utility problem in machine learning, where the machine learning algorithm needs to consider how the scarcity of a critical resource in the deployment environment affects the utility of any learned model. In this paper we focus on local outliers and propose an incremental solution assuming finite memory available. Our experimental results on a variety of data sets show that our solution is well suited to application environments with limited memory (e.g., wireless sensor networks) where the state of the system is changing.