Abstract
Trading surveillance systems screen and detect anomalous trades of equity, bonds, mortgage certificates among others. This is to satisfy federal trading regulations as well as to prevent crimes, such as insider trading and money laundry. Most existing trading surveillance systems are based on hand-coded expert-rules. Such systems are known to result in long developing process and extremely high “false positive” rates. We participate in co-developing a data mining based automatic trading surveillance system for one of the biggest banks in the US. The challenge of this task is to handle very skewed positive classes (< 0.01%) as well as very large volume of data (millions of records and hundreds of features). The combination of very skewed distribution and huge data volume poses new challenge for data mining; previous work addresses these issues separately, and existing solutions are rather complicated and not very straightforward to implement. In this paper, we propose a simple systematic approach to mine “very skewed distribution in very large volume of data”.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Preview
Unable to display preview. Download preview PDF.
References
Fan, W., Wang, H., Yu, P.S., Stolfo, S.: A framework for scalable cost-sensitive learning based on combining probabilities and benefits. In: Second SIAM International Conference on Data Mining, SDM 2002 (2002)
Shafer, J., Agrawl, R., Mehta, M.: SPRINT: A scalable parallel classifier for data mining. In: Proceedings of Twenty-second International Conference on Very Large Databases (VLDB 1996), pp. 544–555. Morgan Kaufmann, San Francisco (1996)
Gehrke, J., Ganti, V., Ramakrishnan, R., Loh, W.Y.: BOAT-optimistic decision tree construction. In: Proceedings of ACM SIGMOD International Conference on Management of Data, SIGMOD 1999 (1999)
Chan, P.: An Extensible Meta-learning Approach for Scalable and Accurate Inductive Learning. PhD thesis, Columbia University (1996)
Author information
Authors and Affiliations
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2004 Springer-Verlag Berlin Heidelberg
About this paper
Cite this paper
Fan, W., Yu, P.S., Wang, H. (2004). Mining Extremely Skewed Trading Anomalies. In: Bertino, E., et al. Advances in Database Technology - EDBT 2004. EDBT 2004. Lecture Notes in Computer Science, vol 2992. Springer, Berlin, Heidelberg. https://doi.org/10.1007/978-3-540-24741-8_46
Download citation
DOI: https://doi.org/10.1007/978-3-540-24741-8_46
Publisher Name: Springer, Berlin, Heidelberg
Print ISBN: 978-3-540-21200-3
Online ISBN: 978-3-540-24741-8
eBook Packages: Springer Book Archive