The nearest-neighbor method is perhaps the simplest of all algorithms for predicting the class of a test example. The training phase is trivial: simply store every training example, with its label. To make a prediction for a test example, first compute its distance to every training example. Then, keep the k closest training examples, where k ≥ 1 is a fixed integer. Look for the label that is m...

Benjamini, Pemantle, and Peres constructed nearest neighbor processes which have predictability profiles that decay faster than that of the simple random walk. Häggström and Mossel found processes with even faster decaying predictability profiles. We prove that rate of decay achieved by Häggström and Mossel is optimal.

We introduce the visible k nearest neighbor (VkNN) query, which finds the k nearest objects that are visible to a query point. We also propose an algorithm to efficiently process the VkNN query. We compute the visible neighbors incrementally as we enlarge the search space. Our algorithm dramatically reduces the search cost compared to existing methods that require the computation of the visibil...

Several researchers proposed using non-Euclidean metrics on point sets in Euclidean space for clustering noisy data. Almost always, a distance function is desired that recognizes the closeness of the points in the same cluster, even if the Euclidean cluster diameter is large. Therefore, it is preferred to assign smaller costs to the paths that stay close to the input points. In this paper, we c...

This paper presents the nearest neighbor value (NNV) algorithm for high resolution (H.R.) image interpolation. The difference between the proposed algorithm and conventional nearest neighbor algorithm is that the concept applied, to estimate the missing pixel value, is guided by the nearest value rather than the distance. In other words, the proposed concept selects one pixel, among four direct...

Linear Discriminant Analysis (LDA) is a popular feature extraction technique in statistical pattern recognition. However, it often suffers from the small sample size problem when dealing with high dimensional data. Moreover, while LDA is guaranteed to find the best directions when each class has a Gaussian density with a common covariance matrix, it can fail if the class densities are more gene...

In this slecture, basic principles of implementing nearest neighbor rule will be covered. The error related to the nearest neighbor rule will be discussed in detail including convergence, error rate, and error bound. Since the nearest neighbor rule relies on metric function between patterns, the properties of metrics will be studied in detail. Example of different metrics will be introduced wit...

Clustering is often formulated as a discrete optimization problem: given a finite set of sample points, the objective is to find, among all partitions of the data set, the best one according to some quality measure. However, in the statistical setting where we assume that the finite data set has been sampled from some underlying space, the goal is not to find the best partition of the given sam...