Prior knowledge

We see an object and we know what it is: a tree, a table, a pen. There is no doubt. Only in a new, unfamiliar environment like a museum, a scientific lab, an industrial workshop, we may see unknown objects and an explanation is need. But in daily life we recognize the objects, even if we visit a neighbor for the first time in his home and see a tree in his garden, a chair in his room or a pen on his table that we have never seen before, we recognize. We may even recognize without finding the word as we will find out when we grow older: “that is a … “, and the word doesn’t pop up, but we know it is a puncher.

Concepts

Recognition is the act of relating an observation to a concepts. Most words, at least almost all nouns, adjectives and verbs are related to concepts. There are also concepts for which the words cannot be directly found or have still to be created. We recognize a baby even before we have given her a name. It is not just “baby”, it is our new-born baby. We recognize her after a day, even when all measurable physical observations have changed. Concepts are usually not sharply defined. The great thing is even that his is for most observations impossible. We may recognize a creature for the deep ocean as an animal even if we would have never thought that an animal could be like that.

Concept learning

Concept learning [1] is a fascinating topic.How do we learn a concept? From examples? From a teacher? The small child suddenly recognizes ‘çows’  after having seen a few and a far from accurate story from his father about farms, fields and milk. The concept pops up in us (is born? is created?) as a generalization from the examples. Suddenly the child knows more than the set of examples. Can knowledge just grow, or is it a transformation of inborn knowledge into the observations around us?

Prior knowledge

Is it possible to extract knowledge from observations without having some prior knowledge? Can the empty mind learn anything, without having a clue of what is significant in the given context, without having a feeling of what is similar and what is different? The pattern recognition literature gives some clear arguments why the answer should be ‘no’  for mechanical learning.

The ugly duckling theorem

Watanabe made perfectly clear by his ugly duckling theorem [2] that all objects are equally different if they are described by a set categorical properties: white, long neck, red bead, short legs, etcetera. Unless we know that these are the relevant ones, all logical combinations of the given properties are equally valid and should be considered as well. If there are n binary properties then the difference between any two objects that are not fully identical is 2^n-1, as shown by Watanabe [2]. So we need some prior knowledge about properties that are relevant.

The no-free lunch theorem

For the case of objects given by continuous features Wolpert showed [3] that unless we know something about the probability density functions of these objects, any recognition system will be as good as any other system. They will all yield arbitrary results. equally good as random guessing.

Prior knowledge is a must

So without prior knowledge it is not possible to learn anything. This leaves us with the painful question: how did it start? How did we start to build knowledge. The preliminary answer, for some sufficient and for others totally unsatisfactory is: prior knowledge is inborn. We are born knowledgeable.