Is pattern recognition about statistics? Well, it depends. If you see as its target to understand how new knowledge can be gained by learning from examples the role of statistics may be disputable. Knowledge lives in the human mind. It is born in the marriage between observations and reasoning. If we follow this process consciously…
The role of densities in pattern classification
Wednesday, October 14th, 2015 at
12:29
Discovered by accident
Tuesday, September 8th, 2015 at
22:17
Some discoveries are made by accident. The wrong road brought a beautiful view. An arbitrary book from the library gave a great, new insight. A procedure was suddenly understood in a discussion with colleagues during a poster session. In a physical experiment a failure in controlling the circumstances showed a surprising phenomenon. Children playing with…
Cross-validation
Wednesday, July 15th, 2015 at
15:13
A returning question by students and colleagues is how to define a proper cross-validation: How many folds? Do we need repeats? How to determine the significance?. Here are some considerations. Why cross-validation? Cross-validation is a procedure for obtaining an error estimate of trainable system like a classifier. The resulting estimate is specific for the training…
Adaboost and the Random Fisher Combiner
Monday, June 8th, 2015 at
11:48
Like in most areas, pattern classification and machine learning have their hypes. In the early 90-s the neural networks awoke and enlarged the community significantly. This was followed by the support vector machine reviving the applicability of kernels. Then, from the turn of the century the combining of classifiers became popular, with significant fruits like adaboost…
Using the test set for training
Tuesday, May 5th, 2015 at
17:07
Never use the test set for training. It is meant for independent validation of the training result. If it has been somewhere included in the training stage it is not independent anymore and the evaluation result will be positively biased. This has been my guideline for a long time. Some students were shocked when I…
My classifier scores 50% error. How bad is that?
Friday, April 3rd, 2015 at
17:27
What error rates can we expect for a trained classifier? How good or bad is a 50% error? Well, if classes are separable, a zero-error classifier is possible. But a very bad classifier may assign every object to the wrong class. Generally, all errors between zero and one are possible: . Much more can be…
Is every pattern recognition problem a small sample size problem?
Monday, February 23rd, 2015 at
14:56
If we want to learn a new concept we may ask for a definition. This might be good in mathematics, but in real life it is often better to get some examples. Let us, for instance, try to understand what despair means. The dictionary tells us that it means ‘loss of hope’. This is just…
Aristotle and the ugly duckling theorem
Monday, November 24th, 2014 at
14:52
We already discussed several times the significance of understanding the Platonic and Aristotelian ways of gaining knowledge. It can be of great help to researchers in the field of pattern recognition in the appreciation of contributions by others, in discussions with colleagues and in supervising students. This may hold for science in general, but it…
Why is the nearest neighbor rule so good?
Monday, October 13th, 2014 at
11:21
Just compare the the new observations with the ones stored in memory. Take the most similar one and use its label. What is wrong with that? It is simple, intuitive, implementation is straightforward (everybody will get the same result), there is no training involved and it has asymptotically a very nice guaranteed performance, the Cover…
There is no best classifier
Friday, September 12th, 2014 at
21:17
Every problem has its own best classifier. Every classifier has at least one dataset for which it is the best. So there is no end to pattern recognition research as long as there are problems that are at least slightly different from all other ones that have been studied so far. The reason for this…
Follow us
