"Modelling natural action selection"

Introduction to the Special Issue "Modelling natural action selection"
Philosophical Transactions of the Royal Society London B
September 2007

A special issue containing fourteen papers on various aspects of modelling action selection in animals and humans. There are multiple methods in the study of biological action selection ("of deciding 'what to do next'"), each approaching the problem from different points of view: be it ethology or psychology studying the biological systems, or artificial intelligence or artificial life attempting to build artefacts based on the biological principles. These differing approaches may be characterised by distinguishing between the analytical and synthetic methods in the behavioural and brain sciences, as proposed by Braitenberg.

The analytical method seeks to describe transitions in behaviour (and to explain why they occur in any given context), either functionally through reference to some concept of utility, or mechanistically through reference to underlying neural systems. It is at the overlap between these two approaches that modelling (the "synthetic" approach - the creation of artificial systems which behave as natural ones) has an increasing influence: "what would it take to build a system that acts in this way?" It is not only formal mathematical models which are of interest, but also, and in cooperation, simulations for which mathematical solutions are intractable or unknown. This usually consists of extracting desired properties to be reproduced through observation, and their characterisation using tools from more traditional experimental sciences. In this way, novel hypotheses may be developed, and empirical data may be analysed from the perspective of the simulation model.

In creating models of action selection (and of other models of biological functioning), there are four important questions which need to be addressed and which should also be bourne in mind during development (quote from paper): "...is the model sufficiently constrained by biological data that its functioning can capture interesting properties of the natural system of interest? Do manipulations of the model, intended to mirror scientific procedures or observed natural processes, result in similar outcomes to those seen in real life? Does the model make predictions? Is the model more complex than it needs to be in order to describe a phenomenon, or is it too simple to engage with empirical data?" Concerning this last question, there is then a trade-off between simplicity for the purposes of analysis and adequate complexity for a sufficient model of the process(es) in question, neither extreme of which is suitable.

The introductory paper (by Prescott, Bryson and Seth) summarises the eight main areas covered by the special issue contributors (optimality of action selection, cortical-basal ganglia substrates, behavioural sequencing, subcortical substrates, disorders, perceptual selection, units of selection, and action selection in social contexts) and provides a discussion of modelling strategies and techniques in general. I hope to summarise a few of these papers in the coming weeks (particularly Botvinicks paper on a model of Fusters 'hierarchies').

Contents of special issue, here. Unfortunately, all of the papers are available by subscription only.

- Introduction. Modelling natural action selection TJ Prescott, JJ Bryson and AK Seth
- Do we expect natural selection to produce rational behaviour? AI Houston, JM McNamara and MD Steer
- The ecology of action selection: insights from artificial life AK Seth
- Compromise strategies for action selection FL Crabbe
- Action selection and refinement in subcortical loops through basal ganglia and cerebellum JC Houk, C Bastianen, D Fansler, A Fishbach, D Fraser, PJ Reber, SA Roy and LS Simo
- Cortical mechanisms of action selection: the affordance competition hypothesis P Cisek
- Towards an executive without a homunculus: computational models of the prefrontal cortex/basal ganglia system TE Hazy, MJ Frank and RC O'Reilly
- Multilevel structure in behaviour and in the brain: a model of Fuster's hierarchy MM Botvinick
- Is there a brainstem substrate for action selection? MD Humphries, K Gurney, and TJ Prescott
- Understanding decision-making deficits in neurological conditions: insights from models of natural action selection MJ Frank, A Scheres and SJ Sherman
- Extending a biologically inspired model of choice: multi-alternatives, nonlinearity and value-based multidimensional choice R Bogacz, M Usher, J Zhang and JL McClelland
- Biologically constrained action selection improves cognitive control in a model of the Stroop task T Stafford and KN Gurney
- Agent-based modelling as scientific method: a case study analysing primate social behaviour JJ Bryson, Y Ando and H Lehmann
- An agent-based model of group decision making in baboons WI Sellers, RA Hill and BS Logan
- Spatial models of political competition with endogenous political parties M Laver and M Schilperoord

10,000

A week or two ago now (I don't know exactly, since I've not been about much to check), this little blog received its 10,000th visitor - which is around 9,900 more than I was expecting when I started just over a year ago, since it was intended as a repository of my thoughts and notes. I hope that there has been something of interest and utility, and I would like to use this opportunity to thank all those who have linked here or left comments - many have led me in directions of research both interesting and useful to my own work.
Paul :-)

Behaviour-based robotics and artificial ethology

The following are quotes from the introductory paragraphs to chapter two of "Behaviour-based Robotics", Ronald C. Arkin, 1998 (MIT Press):

"The possibility of intelligent behaviour is indicated by its manifestation in biological systems. It seems logical then that a suitable starting point for the study of behaviour-based robotics should begin with an overview of biological behaviour. First, animal behaviour defines intelligence. Where intelligence begins and ends is an open-ended question, but we will concede in this text that intelligence can reside in subhuman animals. Our working definition will be that intelligence endows a system (biological or otherwise) with the ability to improve its likelihood of survival within the real world and where appropriate to compete or cooperate successfully with other agents to do so. Second, animal behaviour provides an existence proof that intelligence is achievable. It is not a mystical concept, it is a concrete reality, although a poorly understood phenomenon. Thirdly, the study of animal behaviour can provide models that a roboticist can operationalise within a robotic system. These models may be implemented with high fidelity to their animal couterparts or may serve only as an inspiration for the robotics researcher."

These three points provide the basis for the point of view that animal behaviour has a lot to offer the robotics community (behaviour-based robotics to be specific), and hints at the potential feedback that such work may offer the biologists. Without actually mentioning the term, this description could just as well be applied to artificial (or computational) ethology. The first point I think to be particularly interesting. In my opinion, the working definition of 'intelligence' it introduces is not particularly controversial - however, the implication of the the phrase '...improve its likelyhood of survival...' for cognitive/autonomous robotics is that without some form of actual physical dependancy on the environment (e.g. 'food' - and I hesitantly add, some form of concept of 'life and death' for the agent concerned), intelligence for an artificially created being means nothing (see a related concept in embodiment: organismic embodiment). The second point is one which is generally assumed, but not usually explicitly stated, and something which I think is useful to remind oneself of occasionally. The third point I think is self evident, stated many times, and with plenty of examples in the literature. In fact I think it is the basis for most cognitive robotics work.

The next part of the introduction to chapter two lists two reasons why the robotics community has traditionally resisted the use of the previously mentioned methods of creating artificial agents with 'useful' behaviours (e.g. perceiving and acting in an environment):

"First, the underlying hardware is fundamentally different. Biological systems bring a large amount of evolutionary baggage unnecessary to support intelligent behaviour in their silicon based counterparts. Second, our knowledge of the functioning of biological hardware is often inadequate to support its migration from one system to another. For these and other reasons, many roboticists ignore biological realities and seek purely engineering solutions."

The second point is, I feel, perfectly justified. One only has to consider, for example, the complexity of natural neurons and networks in comparison to the most advanced artificial neural networks which use population-based firing rates, to see that this is true. The first point however, I don't think is necessarily true, especially if one considers that the biological hardware which 'produces' the intelligent behaviour we seek holds many of answers. In this case, an understanding of the 'evolutionary baggage' which produces the biological hardware would be of importance when seeking to understand the intelligent behaviour itself. Or so I think, anyway.

Encephalon #35

I've been slacking recently when it comes to updating the blog... Anyways...

The 35th edition of Encephalon is now up at The Primate Diaries, with 17 of the best neuroscience (and related) posts from the last fortnight. I particularly liked the post from Pure Pedantry with some videos of synapse activation by insertion of AMPA-R into neurons (if I understand correctly, you're much better off reading it for yourself). Fascinating!