On Robots and Psychology

Valentino Braitenberg's Vehicles are often the first lesson in (cognitive) robotics courses as a prime example of the interaction between an agent and its environment, and how complex behaviour does not necissarily imply a complex control architecture (as well as Rodney Brooks' work of course). The fourteen vehicles (or agents) of increasing complexity demonstrate an 'evolution' of behaviours, dependant as much on the environment as on the morphologyand control architecture of the agents themselves. I think the following paragraphs, quoted from the introduction to the book (full reference at end) illustrates how this examination may lead to insight into the biological organisms from which they were inspired (whilst not claiming to 'solve' any of the problems):

I have been dealing for many years with certain structures within animal brains that seemed to be interpretable as pieces of computing machinery because of their simplicity and/or regularity. Much of this work is only interesting if you are yourself involved in it. At times, though, in the back of my mind, while I was counting fibres in the visual ganglia of the fly or synapses in the cerebral cortex of the mouse, I felt knots untie, distinctions dissolve, difficulties disappear, difficulties I had experienced much earlier when I still held my first naive philosophical approach to the problem of the mind. This process of purification has been, over the years, a delightful experience. The text I want you to read is designed to convey some of this to you, if you are prepared to follow me not through a world of real brains but through a toy world that we create together.

We will talk only about machines with very simple internal structures, too simple in fact to be interesting from the point of view of mechanical or electrical engineering. Interest arises, rather, when we look at these machines or vehicles as if they were animals in a natural environment. We will be tempted, then, to use psychological language in describing their behaviour. And yet we know very well that there is nothing in these vehicles that we have not put in ourselves. This will be an interesting educational game.

I like this, as I feel that it represents in some way (albeit more poetically than is generally stated) one of the aims of cognitive robotics as a field: elucidating issues in psychology/neuroscience, via a process (which I have vastly oversimplified here) of model creation on the basis of some biological system, implementation of said model (embodiment in the real world, or simulation thereof), and then evaluation of the resulting system against the original biological system. Hence the emphasis on behaviour - as a means of performing this comparison (since the 'computational substrate' is so obviously different) - which leads to the field of Artificial Ethology.

Ref: Valentino Braitenberg, "Vehicles: experiments in synthetic psychology", 1984

Hammers and Distributed Memory

In “The Feeling of what Happens” (1999), Antonio Damasio describes, among many other things, the distributed nature of memory. The following quote describes an example of how the concept of an object (in this case a hammer) may be represented in the brain (from page 220 of the book):

The brain forms memories in a highly distributed manner. Take, for instance, the memory of a hammer. There is no single place of our brain where we will find an entry with the word hammer followed by a neat dictionary definition of what a hammer is. Instead, as current evidence suggests, there are a number of records in our brain that correspond to different aspects of our past interaction with hammers: their shape, the typical movement with which we use them, the hand shape and the hand motion required to manipulate the hammer, the result of the action, the word that designates it in whatever many languages we know. These records are dormant, dispositional, and implicit, and they are based on separate neural sites located in separate high-order cortices. The separation is imposed by the design of the brain and by the physical nature of our environment. Appreciating the shape of a hammer visually is different from appreciating its shape by touch; the pattern we use to move the hammer cannot be stored in the same cortex that stores the pattern of its movement as we see it; the phonemes with which we make the word hammer cannot be stored in the same place, either. The spatial separation of the records poses no problem, as it turns out, because when all the records are made explicit in image form they are exhibited in only a few sites and are coordinated in time in such a fashion that all the recorded components appear seamlessly integrated.

If I give you the word hammer and ask you to tell me what hammer means, you come up with a workable definition of the thing, without any difficulty, in no time at all. One basis for the definition is the rapid deployment of a number of explicit mental patterns concerning these varied aspects. Although the memory of separate aspects of our interaction with hammers are kept in separate parts of the brain, in dormant fashion, those different parts are coordinated in terms of their circuitries such that the dormant and implicit records can be turned into explicit albeit sketchy images, rapidly and in close temporal proximity. The availability of all those images allows us, in turn, to create a verbal description of the entity and that serves as a base for the definition.

This 'story' of the recall of an abstract concept (by which I mean something which is not explicitly tied to a particular sensory experience) describes a situation where memory, as is generally thought of (recalling objects and events), is fully distributed throughout the brain, and can not be localised in a particular region. This concept has had much supporting empirical evidence found in recent years, including that I've reviewed in last weeks series of posts (which I believe was first proposed by Lashley in the early 20th century).

Part 3 - Areal specificity in the PFC and temporal integration

In this third and final part of my review of this paper by Fuster (reference at the end of this post), the paper turns to look a little more closely at what neural mechanisms are at play in the performance of the central function it is proposed to carry out: temporal integration of information, and more generally, the temporal organisation of behaviour.

The paper then turns to the question of the supposed area specialisations in the cortex: the Network Memory theory predicts that functional subdivisions do not exist, but multiple lesion studies have suggested the opposite, to the extent that some cortical regions are generally accepted to be tied to a particular function. The PreFrontal Cortex (PFC) in particular has been subject to many proposed functional subdivisions, according to functions such as attention, working memory (visual, spatial and verbal), and involvement in episodic and semantic memory. The two points of views thus need to be reconciled.

Fuster addresses this problem by proposing that there is a common underlying process to all of the previously mentioned functions. As has been mentioned, this common process is proposed to be temporal integration. An example of this process in the PFC is given by cells which respond specifically to two temporally separate stimuli which have been associated through experience. These neurons have been demonstrated in monkey studies where the presented stimuli consisted of colours and sounds, so that for a given behavioural goal (which for the monkeys was learning and remembering associations between a colour and a sound, for some reward) cells "...seemed to belong to executive networks that, ..., integrated sounds with colours over time." This example demonstrates a principle with important implications for the concept of working memory, particularly the view of executive functions. Instead of dedicated neural circuitry for executive processes as has been proposed by most psychologically based working memory theories, this example (and other converging evidence) indicates that "...executive working memory seems to be essentially based on the ad hoc activation of executive networks of long-term memory."

From this proposal, it becomes necessary to answer two questions: (1) how are these executive networks activated in a timely fashion for the task at hand; and (2) how is this network (or networks) maintained active for the temporal bridging of separate components?

In order to process perceptual information, some form of modulation is required from the PFC – this being necessary for the context dependant retrieval of information from the posterior cortex (i.e. The perceptual hierarchy of the Network Memory theory). Feedback from these regions would then activate executive representations in the PFC. Together with the influences from sub-cortical regions, and the obvious effect of the external environment on the activation of perceptual and motor memories, this loop may be held responsible for the so called 'monitoring' function of the PFC, where relevant information is activated and made available as and when required. It also addresses the question of 'implementation' of working memory: this mechanism strongly indicates that "[it] is as widely distributed as the long-term memory that supports it." Furthermore, the cooling (known to inhibit brain function in a relatively tightly specified region) of any of these implicated regions in monkeys when performing a behavioural task produces deficits in performance indicative of working memory deficit. This may be explained by viewing such an inhibition as interrupting the 'loops' of recurring activity just mentioned, and thus impairing the monitoring function. In summary then: "Working memory is emerging as a mechanism of temporal integration essentially based on the concurrent and recurrent activation of cell assemblies in long-term memory networks of frontal and posterior cortex."

In the paper conclusion, we are reminded of the evidence from a wide range of disciplines in support of the proposed cardinal function of the PFC being the temporal organisation of behaviour, and temporal integration: anatomical interconnections, and developmental evidence (both in the individual and in evolutionary history). The PFC operates as part of the perception-action cycle, which links the actions of the animal with the environment and its internal neural dynamics.

REF: Fuster (2001), "The Prefrontal Cortex – An update: Time is of the essence", Neuron, vol. 30, pp319-333

Part 2 - The PFC: a temporal organiser

There have been a number of studies in the past (including Endel Tulving, who proposed the view of separate semantic and episodic memory, and the associated HERA model), particularly neuroimaging based, which have observed two things in particular: firstly that the encoding of new memories activates the left PFC more than the right PFC, and secondly that the reverse activation profile is seen when memory is retrieved. This indicates the sort of specialisation that Fuster moves away from. An argument is thus provided to counter this: "...it is not altogether clear that the asymmetric activations of the two prefrontal cortices in encoding and retrieval are attributable to their differential involvement in these two cognitive operations. The apparent functional dissociation of right and left PFC may be a product of the subtractive method if the material utilized to test the two operations carries a different semantic load... in both encoding and retrieval tasks, the executive memory networks of lateral PFC may be activated inasmuch as the tasks consist of temporally integrative acts based on internal or external contingencies." In summary, the lateral PFC's most important function is the temporal organisation of behaviour.

Whereas routine behaviours sequences may be made up of chained behaviours, where one action leads to the next, there is no temporal contingency required - i.e. no planning. However, when ambiguity or uncertainty is introduced, such temporal planning ("cross-temporal contingencies") is required. Hence the role of the PFC (in delay tasks, for example, where there is at least one ambiguity that needs to be resolved) in temporal integration. The figure below, taken directly from the paper (figure 4 in paper), gives examples of these two events: in the top case, a routine sequence of behaviours occurs, and in the bottom, a complex or novel sequence. In this latter case, any given act is contingent on the goal, the plan, and other acts - this is the hypothesised role of the lateral PFC: the temporal organisation of behaviour.


Given this role for the lateral PFC, working memory is proposed as the first "temporal integrative function" of the PFC which has been verified in primates using electrophysiology. These experiments, which used delayed response tasks (and occurred in the early '70s), have identified four properties of neural firing related to spatial memory (thus also working memory):
(1) it was absent after stimuli that did not call for prospective action;
(2) it was absent in the mere expectation of reward;
(3) it was correlated with the accuracy of the animal's performance; and
(4) it could be diminished or aborted by distracting stimuli occurring in the delay period.

These properties strongly indicate "retention of the memorandum" (that which is to be remembered), which is dependant on the future need for action. "In sum, considerable evidence from several methodologies supports a prospective role of the PFC, in addition to its retrospective role of working memory. Ingvar (1985) dubbed that prospective role the 'memory of the future'". The retention of the memorandum could then allow the activation of the necessary motor functions (so called motor attention or preparatory set), thus priming the system as a whole for the execution of the required actions: “A temporally symmetric and complementary function of preparatory set, or 'motor attention' would activate the network's motor components and thus prime executive systems for the anticipated action.”

The final part of this paper review will be posted tomorrow, on more specific cortical mechanisms involved in this proposed function of the PFC...

REF: Fuster (2001), “The Prefrontal Cortex – An update: Time is of the essence”, Neuron, vol. 30, pp319-333

Part 1 - The PFC: anatomy and the network memory theory

The PreFrontal Cortex (PFC) is widely implicated in many 'high-level' cognitive functions, from working memory to planning, and others. It is frequently decomposed into a number of subregions, each supposedly specialised in some way or another for particular tasks. It is however difficult to establish these claims for certain given that current neuroimaging techniques do not have adequate spatial and temporal resolution to examine the real-time workings of these very small subregions of the PFC. All that can be done is to collect a wide variety of evidence from various methodologies in support of the theory. This is what Fuster does in this paper (ref at end): neuroanatomical, neuroimaging, single-cell recordings and behavioural studies are all reviewed in support of his theory of PFC functioning. His view is that the PFC essentially performs the function of temporally integrating information in order to achieve behavioural goals. This view was covered briefly covered in a previous post, but a number of details were missing from that short paper, which this paper attempts to address.

Fuster considers the PFC as being at the apex of the motor hierarchy in the Network Memory theory. As such, the supposed widely distributed network nature of the cortex in general is of great importance (quote "Any hypothetical modularity of the PFC is functionally meaningless if taken out of wide-ranging networks that extend far beyond the confines of any given prefrontal area."), and that is where the paper starts - the placement of the PFC in the cortex, with regard to neural connectivity. A summary of these connections by means of a quote (text in italics added): "The PFC is connected with the brainstem, the thalamus, the basal ganglia, and the limbic system (hypothalamus in particular). Much of that connectivity with subcortical structures is reciprocal. Especially well organized topologically are the connections between the PFC and the thalamus. The prefrontal connections with the mediodorsal thalamic nucleus have been used as a criterion for identifying the PFC in a wide variety of species." In terms of cortical connectivity, the PFC has connections to the cortices of association, but not with the primary sensory or motor cortices - which given the Network Memory theory's placement of the PFC at the top of a hierarchy, makes sense.

The PFC may be divisible into three main sub-regions: medial, orbital, and lateral. Each of these is 'connected' to itself and to each other. Damage to each of these three sub-regions typically produce distinct behavioural deficits. Damage to the orbital PFC (such as that sustained by the famous Phineas Gage) results in behaviour which is impulsive, an inability to inhibit instinctual behaviours, and typically exhibit a severe attention disorder (inability to withstand distraction). Damage to the medial part of the PFC generally result loss of spontaneity, and difficulty in the initiation of movements and speech. However, it is damage to the lateral PFC which produce the most characteristic cognitive deficit (according to Fuster). For humans, the most notable deficit is in the ability to plan and carry out sequences of actions which manifests itself in a difficulty in both speech and behaviour, and the difficulty to initiate and perform sequences in an orderly manner. Fuster points to this in particular to be a strong indication that the lateral PFC "plays a crucial role in the organization and execution of behavior, speech, and reasoning."

After reviewing the basics of the Network Memory theory (which may be seen in this post), the paper then goes on to describe in broad terms how the PFC fits in, and what more precisely its role is. The low levels of the motor hierarchy are located in the primary motor cortex, the so-called phyletic motor memory, which defines movements by muscle actions. Higher up in the motor hierarchy, these may be grouped, such that movements are defined by goals and/or trajectories. These higher levels are located in the PFC, and include, for instance, "elementary linguistic structures". The lateral PFC in particular "appears to harbour networks representing schemas, plans, and concepts of action." However, as sequences become over-learned and automatic, their representations 'move' towards lower executive levels: i.e. as the 'novelty' of an action decreases (or becomes more routine), less planning is required, and so the representation may 'drop down the hierarchy' (Fuster mentions that they may be organised entirely in sub-cortical structures). Despite this, if ambiguities in the action remains, then the involvement of the PFC remains necessary, in order to "integrate events in the temporal domain". A note is made that despite these functions being at the top of the hierarchy, they are not necessarily organised hierarchically: they are to some extent at least organised heterarchically. This is an important point, as it is an effort to acknowledge that the PFC does not perform top-down serial processing, as is often implied by a strict hierarchical structure.

This paper review continues tomorrow with a closer look at the function of the PFC...

REF: Fuster (2001), "The Prefrontal Cortex – An update: Time is of the essence", Neuron, vol. 30, pp319-333

The Prefrontal Cortex: Time is of the Essence

In this paper, Prof. Joaquin Fuster details the evidence he has from a wide range of disciplines in support of his proposed function for the prefrontal cortex. Since my review of the paper is itself quite lengthy, I have split it up into three parts, which will be posted here over the next three days. These three parts are as follows:

(1) The PFC: anatomy and the network memory theory

(2) The PFC: a temporal organiser

(3) Areal specificity in the PFC and mechanisms involved in temporal integration

Reference: Fuster (2001), “The Prefrontal Cortex – An update: Time is of the essence”, Neuron, vol. 30, pp319-333