Avoiding Anomalies in Data Stream Learning

Warning

This publication doesn't include Faculty of Medicine. It includes Faculty of Informatics. Official publication website can be found on muni.cz.
Authors

GAMA Joao KOSINA Petr ALMEIDA Ezilda

Year of publication 2013
Type Article in Proceedings
Conference Discovery Science, Proceedings of 16th International Conference DS 2013
MU Faculty or unit

Faculty of Informatics

Citation
Web http://link.springer.com/chapter/10.1007%2F978-3-642-40897-7_4
Doi http://dx.doi.org/10.1007/978-3-642-40897-7_4
Field Informatics
Keywords Data Streams; Rule Learning; Anomaly Detection
Description The presence of anomalies in data compromises data quality and can reduce the effectiveness of learning algorithms. Standard data mining methodologies refer to data cleaning as a pre-processing before the learning task. The problem of data cleaning is exacerbated when learning in the computational model of data streams. In this paper we present a streaming algorithm for learning classification rules able to detect contextual anomalies in the data. Contextual anomalies are surprising attribute values in the context defined by the conditional part of the rule. For each example we compute the degree of anomaliness based on the probability of the attribute-values given the conditional part of the rule covering the example. The examples with high degree of anomaliness are signaled to the user and not used to train the classifier. The experimental evaluation in real-world data sets shows the ability to discover anomalous examples in the data. The main advantage of the proposed method is the ability to inform the context and explain why the anomaly occurs.
Related projects:

You are running an old browser version. We recommend updating your browser to its latest version.

More info