The Nearest Neighbor (NN) rule is a classic in pattern recognition. It is intuitive and there is no need to describe an algorithm. Everybody who programs it obtains the same results. It is thereby very suitable as a base routine in comparative studies. But who invented it? Marcello Pelillo looked back in history and tried…
Classification Archives
Who invented the nearest neighbor rule?
Tuesday, January 28th, 2014 at
13:58
The error in the error
Tuesday, July 23rd, 2013 at
15:07
How large is the classification error? What is the performance of the recognition system? At the end this is the main question, in applications, in proposing novelties, in comparative studies. But how trustworthy is the number that is measured, how accurate is the error estimate? The most common way to estimate the error of a…
Classifying the exception
Monday, May 27th, 2013 at
00:00
Exceptions do not follow the rules. That is their nature. Humans know how to handle them. Can that be learnt? Learning a rule One of the first real world datasets I had to handle consisted of the examination results of the two-year propedeuse in physics. Students passed or failed depending on their scores for 15…
Recognition, belief or knowledge
Tuesday, May 14th, 2013 at
00:00
Recognition systems have to be trained. An expert is necessary to act as a teacher. He has to know what is what. But … Does he really know, or does he just believe that he knows? Or, does he know that he just believes? And, does he know how good his belief is? Nils Nilsson,…
Pattern recognition and neural networks, Kuhn and Popper
Monday, May 6th, 2013 at
00:00
Is the neural network model good for pattern recognition? Or, is it too complex, too vague, too clumsy to be of any use in performing applications or in building understanding? The relation between the pattern recognition community and these questions have always been very sensitive. Its history is also interesting for observing how science may…
Peaking summarized
Monday, April 29th, 2013 at
00:00
Pattern recognition learns from examples. Thereby, generalization is needed. This can only be done if the objects, or at least the differences between pattern classes have a finite complexity. That is what peaking teaches us. We will go once more through the steps. (See also our previous discussions on peaking, dimensionality problems and Hughes’ phenomenon)….
Trunk’s example of the peaking phenomenon
Monday, April 15th, 2013 at
00:00
In 1979 G.V. Trunk published a very clear and simple example of the peaking phenomenon. It has been cited many times to explain the existence of peaking. Here, we will summarize and discuss it for those who want to have a better idea about the peaking problem. The paper presents an extreme example. Its value…
A crisis in the theory of pattern recognition
Monday, April 1st, 2013 at
00:00
The Russian scientist A. Lerner published in 1972 a paper under the title: “A crisis in the theory of Pattern Recognition”. This is definitely a title that attracts the attention of researchers interested in the history of the field. What was it to have appeared as the crisis? The answer is surprising, in short it…
The curse of dimensionality
Monday, March 25th, 2013 at
00:00
Imagine a two-class problem represented by 100 training objects in a 100-dimensional feature (vector) space. If the objects are in general position (not by accident in a low-dimensional subspace) then they still fit perfectly in a 99-dimensional subspace. This is a ‘plane’, formally a hyperplane, in the 100-dimensional feature space. We will argue that this…
Generalization by dissimilarities
Monday, January 7th, 2013 at
00:00
Dissimilarities have the advantage over features that they potentially consider the entire objects and thereby may avoid class overlap. Dissimilarities have the advantage over pixels that they potentially consider the objects as connected totalities, where pixels tear them apart in thousands of pieces. Consequently, the use of dissimilarities may result in better classification performances…
Follow us
