Simplicity Theory

Simplicity, Complexity, Unexpectedness, Cognition, Probability, Information

by Jean-Louis Dessalles
(created 2008.12.31)

(updated 2010.02.17)

Conceptual Complexity

Unexpectedness requires simple concepts

The complexity of situations is the minimal description that makes them unique. The window below may be felt as simple (it is a window, quite a simple thing, isn’t it?). It is not! Kolmogorov complexity in general and also in the particular context of Simplicity Theory deals with minimal descriptions of objects, not of classes. For instance, the complexity of 131072 is not the complexity of the class of 6-digit numbers, nor of the class of even numbers. It is the minimal amount of information thanks to which the observer can individuate the actual number (e.g. by saying that it is 2^17). The complexity of this window is not small, because we still need to individuate it from all other windows.

The fact that the above object is a window can be used as a starting point to compute both generation complexity C_w and description complexity C.

Generation complexity C

Let us call d the selection operation within a class of objects. Generation complexity may be computed through the computation sequence f*d*s :

C_w(f*d*s) = C_w(f) + C_w(d|f) + C_w(s|f&d)

f means ‘windows’ (or ‘fenêtres’)), i.e. all objects that the observer would accept as being windows (fortunately, such a set needs not to be constructed). The complexity of generating these objects is zero: C_w(f) = 0 (windows are supposed to exist). If there are N windows in the world (or in a reference set given by the context, e.g. all the windows in the building) and if the actual one was picked at random in the course of some process leading to situation s, then C_w(d|f) amounts to log₂ N. We get:

C_w(f*d*s) = log₂ N + C_w(s|f&d)

where C_w(s|f&d) is the complexity of everything about the situation excluding the object itself (e.g. spatio-temporal precisions).

Description complexity C

The description complexity of s may go through the same computation sequence f*d*s. The computation is however slightly different.

C(f*d*s) = C(f) + C(d|f) + C(s|f&d)

The first term is non-zero, this time. The best way to characterize the class of all windows is to mention feature ‘window’ (noted f as well). So the computation of C_w(f) requires to consider what is usually called the ‘extension’ of the window concept, whereas the computation of C(f) requires to consider what is usually called the ‘intension’ of the window concept. Note, however, that extension may remain highly fuzzy without preventing from getting a useful estimate of C_w(f) (especially for large sets, thanks to the presence of the logarithm).

C(f) may be called conceptual complexity. Some concepts are more popular than others, and are thus simpler. For most people, for instance, the giant Panda (Ailuropoda melanoleuca, here shown at the San Diego zoo) is a simpler concept than the Arabian babbler (Turdoides squamiceps). Imagine that we store our concepts in lists or trees or graphs; then the complexity of a given concept can be assessed by the logarithm of the rank of the concept in a list or the length of the shortest path to the concept in a graph). An alternative (somewhat crude) way to get an idea of the complexity of concepts consists in ranking them according to the number of hits on a Web search engine (I got 69500 hits for Ailuropoda melanoleuca and 30900 for Turdoides squamiceps), and then to take the logarithm of the rank.

(photo Quentin Dessalles)	(from there)

C(f) is the description cost one has to ‘pay’ when introducing feature f. In our window example, the term C(d|f) is still significant. It may amount up to log₂ N, if one follows the algorithm: ‘number all windows in the world (e.g. by reverse making date) and indicate the logarithm of the rank’. Note that this may make C(s) > C_w(s) and thus unexpectedness U(s) negative.

To do better, one may accumulate features f_i:

C(Pf_i*s) = C(f₁) + C(f₂|f₁) + C(f₃|f₁&f₂) ... + C(d|&f_i) + C(s|&f_i&d)

until one reaches uniqueness: C(d|&f_i) = 0. For most situations, the complexity of defining features C(f₁) + C(f₂|f₁) + C(f₃|f₁&f₂) ... is no less than C_w(d|&f_i), which means that at the end the object does not contribute to unexpectedness U(s) = U(s|d). Interesting objects are those for which one observes a complexity drop.

Note that relevant features can be defined as such when C(Pf_i*s) does not increase when they are introduced (and correlatively, irrelevant features produce a complexity increase).

Bibliography

Dessalles, J-L. (2008). La pertinence et ses origines cognitives - Nouvelles théories. Paris: Hermes-Science Publications.

Dimulescu, A. & Dessalles, J-L. (2009). Understanding narrative interest: Some evidence on the role of unexpectedness. In N. A. Taatgen & H. van Rijn (Eds.), Proceedings of the 31st Annual Conference of the Cognitive Science Society, 1734-1739. Amsterdam, NL: Cognitive Science Society.

Back to the Simplicity Theory page