Example-Based DB-Outlier Detection from High Dimensional….pdfVIP

DEWS2007 M4-4 Example-Based DB-Outlier Detection from High Dimensional Datasets Yuan LI† and Hiroyuki KITAGAWA† ,†† † Graduate School of Systems and Information Engineering †† Center for Computational Sciences University of Tsukuba Tennoudai 1–1–1, Tsukuba, Ibaraki, 305–8573 Japan Abstract Outlier detection is an important problem that has applications in many fields. High dimensional datasets are common in such applications. Among the existing outlier detection methods, Distance-Based outlier (DB-Outlier) detection is one of the most generalizable and simplest approaches. It finds outliers by calculating distances between data points. However, in high dimensional space, data distribution is sparse, so every data point becomes a good outlier candidate. It has been shown that meaningful outliers are likely to be identified by exam- ining the behavior of the data in low dimensional projections. On the other hand, Example-Based outlier detection method is promising in discovering the hidden user view of outliers. In this paper, we present a new method to detect DB-Outliers in high dimensional datasets based on user examples. The method finds a subspace where user examples are outstanding more significantly than in any other subspaces, and reports outliers detected in this subspace. Key words Outlier, DB-Outlier, High-dimensional Data, Example Many real applications process high dimensional datasets 1. Introductio