Technical note: A nonparametric outlier rejection scheme

Abstract

Experimental data always contains measurement errors (or noise, in signal processing). This paper is concerned with the removal of outliers from a data set consisting of only a handful of points. The data set has a unimodal probability distribution function, the mode is thus a reliable estimate of the central tendency. The approach is nonparametric; for the data set (x_i, y_i) only the ordinates (y_i) are used. The abscissa (x_i) are reparametrized to the variable i = 1, N.The data is bounded using a calculated mode and a new measure: the mean absolute deviation from the mode. This does not seem to have been reported before. The mean is removed and low frequency filtering is performed in the frequency domain, after which the mean is reintroduced.