Lessons for Symbolic and Sub-Symbolic Architectures from Biology

Notes on "Symbolic and Sub-Symbolic representations in computational models of human cognition: what can be learned from biology?", T.D. Kelley, Theory and Psychology, vol 13(6), pp 847-860, 2003

There has been considerable debate concerning the role of symbols in the human cognitive architecture: does it use symbols as a representation of knowledge, or does it use distributed representations of knowledge. This paper proposes the third option - that of a hybrid between the two types of system. Lessons from biology are examined and the ACT-R architecture is proposed as the happy medium. The paper first reviews symbolic and sub-symbolic representations separately before discussing their hybrid in ACT-R. The biological evidence supporting this is then presented before the concluding remarks.

Classic cognitive psychology has argued that knowledge is represented as a series of symbols. This idea may be understood by using the metaphor of a computer. In its most basic terms, a computer performs an input-process-output function. The input may take the form of a series of symbols, which are representations of other concepts or constructs. They may be manipulated using a predetermined set of instructions, which then generates an output symbol or symbols. Besides the relationships between symbols which are explicitly supplied, symbolic relationships may also be inferred, however the process of making such inferences is complex and difficult to implement computationally.

Sub-symbolic systems have traditionally supported by those in the field of artificial intelligence, particularly those in the connectionist 'movements'. They are most often associated with artificial neural networks as metaphors of biological neural networks. The artificial neurons (or perceptrons in some implementations) operate in parallel to recognise a given input, by adjusting the weights between the individual nodes. Such a network of nodes may thus be considered an autonomous learning system - which is one of its greatest strengths.

Each of these two systems thus shows certain advantages in the study of the human cognitive system - the sub-symbolic system as an autonomous learning system (though a predetermined training algorithm is needed) and the symbolic system as a means of easily representing complex relationships (though it has difficulties with gaining the knowledge in the first place - the symbol grounding problem). It may be considered that the two approaches as opposite ends of a single continuum. Keeping this in mind for the next part of the paper, it may be viewed that sub-symbolic systems recognise inputs, which then get passed to the symbolic system. First, a summary of the differences:
· symbolic systems process in series, sub-symbolic systems operate in parallel.
· sub-symbolic systems have distributed knowledge, whereas symbolic systems do not (e.g. 4+5 is expicitly represented in a symbolic system at a particular location, but not in its sub-symbolic counterpart).
· sub-symbolic systems learn to recognize inputs, and respond to the recognised input in accordance with learning rules, whereas symbolic systems are not concerned with the recognition of the input, only with the manipulation of the symbols following the recognition.

From these last points, the benefit of a hybrid system between the two methods may be seen, provided that the advantages of both are combined. This brings the paper to the discussion of ACT-R, which is a hybrid, or integration, of sub-symbolic and symbolic systems (developed by Anderson and Lebiere, 1998). It has a production system architecture, where the main type of processing occurs within an if-then format - including a declarative memory (memory for facts) and a procedural memory (a skills memory which for us is not easily verbalised). In ACT-R, each symbolic component is linked to underlying sub-symbolic processes, which are continuously varying and which operate in parallel, while the symbolic part operates in series. However, while this hybrid system takes the advantages of both individual methods, it is still prone to the deficiencies of each. So, much of the 'knowledge' of the architecture is essentially hard-coded by the programmer, and not learned by the sub-symbolic network. Later versions of the ACT-R architecture have started to incorporate perceptual modules which have started to perform this task, thus reducing this particular deficiency. Hybrid architectures in general have become more and more popular as a way of overcoming the shortcomings of the individual approaches.

The rest of the paper is devoted to presenting the biological (human) cognitive system and using this information to provide support for the symbolic sub-symbolic hybrid architecture. Firstly, the biology and capability of the human cognitive system is examined, then these aspects of the cognitive system across different species.

Again using the idea of a continuum, the human cognitive system has at its highest level a symbol processing system, supposedly located in prefrontal cortex regions of the brain. This region has long been seen as the highest 'level' of the brain, representing "the zenith of human cognitive capabilities". Further to this, it (the neocortex) has developed on top of phylogenetically older and simpler systems/brain regions. As an example, language is a symbolic system, which is known to heavily involve the frontal regions - damage to these regions result in severe impairment of language capabilities. At the other end of the continuum, the lowest cognitive mechanism is the reflex, which are the result of the most basic neural networks where the synaptic weights have been set over the course of evolution (this idea has been discussed in a previous post).

Similar to the continuum of simple to complex processes within a single human brain, a continuum exists across different species within the animal kingdom, from single celled organisms and those with the simplest of nervous systems (which could potentially be replicated by a sub-symbolic architecture), to mammals and primates. The learning capabilities of insects may be described as being associative, which is the simple association of a stimulus with a response. This type of learning has been frequently criticised as not being detailed enough to support complex relationships - although this seems at odds with the comments of Joaquin Fuster (reviewed in a previous post). It is on this basis that the need for symbolic systems is proposed. However, when using this, the possibility of coming up against the 'Chinese room argument' (Searle, 1979) becomes likely - i.e. the 'blind' manipulation of symbols without knowing their true meaning.

The conclusions of the paper essentially reiterate the point that a hybrid between symbolic and sub-symbolic approaches to cognitive architectures appears to be the best approach, and have as such been gaining popularity. The SOAR architecture, a fully symbolic architecture, has recently been supplemented with sub-symbolic features - activation levels (as reviewed in a previous post) to great effect. One argument however that arises is that sub-symbolic architectures will eventually 'catch-up' with their symbolic counterparts, and so should not be considered at the bottom of the hierarchy. This is acknowledged to be possible by the author of the paper, although it is noted that traditional connectionist approaches (by which I assume he means multilayer perceptrons) have been shown to be inadequate for representing complex symbolic relationships (by Fodor and Pylyshyn, 1988). Having said this, the symbolic approach still holds numerous attractive benefits to the study of complex cognition, especially when used in combination with a sub-symbolic system.