Disk Aware Discord Discovery: Finding Unusual Time Series in Terabyte Sized Datasets
Yankov, D.
Keogh, E.
Rebbapragada, U.
Univ. of California, Riverside;
This paper appears in: Data Mining, 2007. ICDM 2007. Seventh IEEE International Conference on
Publication Date: 28-31 Oct. 2007
On page(s): 381-390
Location: Omaha, NE,
ISSN: 1550-4786
ISBN: 978-0-7695-3018-5
INSPEC Accession Number: 9894667
Digital Object Identifier: 10.1109/ICDM.2007.61
Current Version Published: 2008-03-12
Abstract
The problem of finding unusual time series has recently attracted much attention, and several promising methods are now in the literature. However, virtually all proposed methods assume that the data reside in main memory. For many real-world problems this is not be the case. For example, in astronomy, multi-terabyte time series datasets are the norm. Most current algorithms faced with data which cannot fit in main memory resort to multiple scans of the disk/tape and are thus intractable. In this work we show how one particular definition of unusual time series, the time series discord, can be discovered with a disk aware algorithm. The proposed algorithm is exact and requires only two linear scans of the disk with a tiny buffer of main memory. Furthermore, it is very simple to implement. We use the algorithm to provide further evidence of the effectiveness of the discord definition in areas as diverse as astronomy, Web query mining, video surveillance, etc., and show the efficiency of our method on datasets which are many orders of magnitude larger than anything else attempted in the literature.
Index
Terms
Available to subscribers and IEEE members.
References
Available to subscribers and IEEE members.
Citing Documents
Available to subscribers and IEEE members.
Cart
