* Department of Computer Science and Electrical Engineering
University of Queensland, 4072, Australia
btonkes@csee.uq.edu.au

abstract

With advances in computational techniques, it has become possible to model aspects of the origin and subsequent evolution of languages. One of the more intriguing aspects of language evolution is the emergence of syntax. While many species have evolved forms of signalling systems, humans are seemingly alone in the use of recursive compositional structures that result in ``essentially infinite'' languages. The complexity of human languages raises the question of how human infants acquire language, particularly since the infant learner can never observe the entire language. Many linguists argue for a strong form of innate linguistic endowment that provides the human infant with knowledge about the universal structures of human languages. An alternative view emphasises the point that languages themselves can act as complex adaptive systems, which evolve to their human ``hosts''.

Kirby (1999a, 1999b) gives a compelling demonstration of syntactic structures emerging from a population of agents. Despite the absence of phylogenetic adaptation in the population, a compositional language emerges as a result of the dynamics of language acquisition. Kirby proposes that individuals' languages can be described in terms of replicators (corresponding to grammatical rules or rule-sets) that compete for survival, with the more general replicators having a greater chance of being learned. Thus, languages evolve towards forms that consist of broad, compositional rules where every utterance provides an opportunity to learn the general rule.

Although there is no phylogenetic adaptation in the course of Kirby's simulations, the model incorporates phylogenetic adaptation implicitly in the design of the individuals' language learning mechanisms. That is, the starting point of the simulations is a population of individuals who are innately endowed with a particular learning mechanism. It seems to us that the chosen induction algorithm is highly biased towards language-like, compositional structures. Although Kirby highlights the importance of languages as adaptive systems that adapt to their human hosts, inherent in his choice of learning algorithm is a strong form of language-specific learning bias. Kirby's simulations demonstrate a set of biases that are sufficient for the emergence of compositional syntax. In this study we wish to consider their necessity. That is, are strong domain-specific learning biases required for compositional language to emerge?

In previous work (Tonkes, Blair and Wiles, 2000) we have considered communication between a pair of recurrent neural networks which provide a more general-purpose learning mechanism than that used by Kirby (though like any learning algorithm, some degree of bias is inherent). The two networks try to communicate a ``concept'' over a symbolic channel. The encoder network sends a sequence of symbols for each concept, which the decoder network receives and processes back into a concept. Using this framework, we have shown how a language can evolve to mediate opposing biases between encoder and decoder, and how language evolution can facilitate learning by adapting towards the forms that exploit the weak biases of a general purpose learner.

In the present study, we investigate the extent to which domain-specific biases are required to replicate Kirby's (1999a) results. We use the same basic design, but substitute our learner (recurrent neural network) and task. The initial population consists of recurrent encoder networks with random weights arranged in a ring. Each of these networks map concepts in the unit interval to sequences of symbols. Following Kirby, in each generation one network is replaced by a new individual which is subsequently trained on the language of its neighbours. As new individuals are introduced to the population, the overall language of the population gradually changes so that only the learnable forms persist.

Although populations converge to languages that are easily learnable, the evolved languages are degenerate - the networks use the same message for each meaning. Attempts to enforce language expressivity result in the non-convergence of the population. Consequently, we investigate the constraints that are necessary in a learning mechanism for the emergence of language that is both expressive, and consistent across a population.

 Conference site: http://www.infres.enst.fr/confs/evolang/