Monday, February 26, 2007

Encephalon #17

The 17th Edition of Encephalon is now up at Pure Pedantry!

A particular post to note is one by Chris at Developing Intelligence, which gives an introduction and overview to the role of Dopamine in the brain. Extremely interesting reading.

Events seem to have conspired to prevent me posting over the last two weeks: a situation I hope to remedy (properly) in the next few days...

Monday, February 12, 2007

Encephalon #16

Encephalon #16 is now up at Mind Hacks!

A few of my picks from this edition:

- Advancements in Neuroprosthetics at FML: reviews some recent developments in brain-electronic interfaces.
- Towards A Neural Account of Decision Making Biases at Developing Intelligence: a review of a paper by McClure et al on the neural substrate of decision making (which I think is related to a recent post of mine on the relationship between emotion and decision making) - in the briefest of summaries, the findings essentially supported the view that prefrontal regions are important in long-term decision making, and sub-cortical regions were more important in making short-term and reward related decisions.
- The Mental Number Line at The Phineas Gage Fan Club: a fascinating look at the spatial encoding of numbers in the brain

Sunday, February 11, 2007

A review of Just Science Week

By way of a final post for the week of science challenge, I have decided to review my previous six posts. Not strictly speaking a paper review, or a 'professional' opinion (which I believe was the goal of posting this past week), but something I think would be interesting - a review of reviews - and make a nice conclusion for the past week. (And I haven't got round to reading another paper - man-'flu has struck once again...).

On Monday, a paper by Bechara et al. on the relationship between emotion and decision making was reviewed (the somatic marker hypothesis). The study essentially found evidence supporting the view that emotion is a central part of decision making, as it's use provides a filter-like function which reduces the decision space and enables decisions to be taken in a time-frame suitable for action in the real world. On Tuesday, a paper on brain based devices by Krichmar and Edelman was looked at. This paper provided an introduction to this work, presenting the principles of building devices (robots) according to neurobiological rather than computational principles. These principles have formed the basis of the Darwin series of robots which are capable of complex learning and completion of tasks - all of which are self learned without explicit pre-programming. Wednesday saw the review of a maze task often used in rat studies to assess spatial memory: The Morris Water Maze. As mentioned, this maze has also recently found use in robotics studies for a similar purpose, as it provides a challenge which involves the interaction of sensory and motor systems, as well as the requirement for memory - all of which are required for an autonomous agent. The posts for Thursday and Friday were somwhat shorter, with Thursday providing a brief discussion on why using artificial agents in the study of biological systems may be useful (something I hope to expand on in a future post), and Friday with a brief review of a paper by Sane et al on the presence of a gyroscope-like system in moths which enable them to achieve a steady flight in the absence of visual cues, which I use as a simple example of the importance of physical implementation of a system as well as the control system. Finally, yesterdays post provided a brief introduction to a tutorial on Humanoid Robotics by the Idaho National Laboratory, which I think is well worth a read.

So, if there is a theme to be found among these posts, it would be a review of work from a wide range of disciplines and their applications to cognitive robotics. I think two things can be said for this: firstly that if one wants to maintain the biological plausibility of a cognitive model, one must take into account the necessities and limitations of the biological system(s) of interest; and secondly, that expanding one's horizons and examining the work of others in different (sometimes completely different) disciplines can provide you ideas and inspiration which would be hard to come by in any other way. That's the hope of my work... hopefully... sometime in the future... :-s

Finally, I'd like to thank all the contributors to the Just Science Week for their interesting and informative posts, and I hope that next year will be just as good. Although I hope that then, I'll be able to keep up with the one-paper-review-a-day thing.

Saturday, February 10, 2007

An Overview of Humanoid Robotics

Through idle surfing of the 'net, I came across a tutorial on Humanoid Robotics, their historical and current development, and potential scenarios for the the future. It's over at the homepage of the Idaho National Laboratory. Using a quote from the introduction explaining what the tutorial covers:

"This site traverses a wide variety of Humanoid Robotics projects throughout the world, explaining the diverse goals of the field and why humanoid robots are uniquely suited to meet these goals. As we review successes and failures in the field, we provide a contextual backdrop for understanding where humanoid research began, the dilemmas it currently struggles with, and where it may take us in the future. Imagination is the bow from which the technology, science and art of Humanoid Robotics takes flight. As we try to discern where the bow is aimed, the paper also asks whether we are ready for the changes that will follow."

I have to admit that I havent read all of it yet, but it seems at first blush to be an interesting read if nothing else. So far, I particularly like the section entitled "Building Intelligence from the bottom up", which provides a very brief overview of the approaches which are taken in attempting to create artificial intelligence, such as artificial neural networks, reinforcement learning, and genetic algorithms - and even their integration:

"For instance, a designer can use a neural network to implicitly encode low-level motor control for an arm-reaching behavior and then use reinforcement learning to train the humanoid when to reach and when to grasp. If the humanoid still struggles, the designer might, for instance, optimize behavior using a genetic algorithm to tweak parameters controlling rotational torque. "

Friday, February 09, 2007

Biological Gyroscopes

Another short post today - and from another story found on the BBC - on the discovery of a gyroscope-like mechanism in moths to aid in the maintenance of in-flight balance.

As is well known, moths are most active at night, or at least during those periods of the day (or rather, night) which have the lowest levels of light. As such, when in flight, it would be near impossible for them to use visual landmarks as a reference for the purpose of stabilisation. So, how is this in-flight stabilisation achieved? According to Sane et al (Published in Science magazine, vol 315, p735), moths have what is in effect a gyroscopic system in their heads. The mechanism works roughly as follows: using an organ in their heads (Johnston's organ), the moths are able to detect their body position in relation to their antennae position, which apparently remains stationary during flight, thus providing what is in effect a fixed reference point. This was demonstrated by removing the antennae, resulting in the moths displaying very irregular flight patterns, bumping into walls and flying backwards. By merely 'glueing' the antennae back on, normal flight behaviour resumed. A somewhat cruel but very effective demonstration.

Coming from my point of view, I think this is an elegant example of how the morphology of an agent has an effect on behaviour, even if the required sensory system is in place. Thus here, the sensor (Johnston's organ) does not contribute adequately to overall agent behaviour if the body shape (antennae) is wrong - thus demonstrating the importance of not only a well designed control system, but also the correct type of physical implementation which complements this control system. So essentially, embodiment a necessity, not an extra.

Thursday, February 08, 2007

A brief discussion of the justification of the use of Artificial Agents in studying Biological Processes

When studying humans and animals, it is necessary to take experimental precautions to ensure that the article of interest is actually studied, and no other. This experimental isolation in practice is impossible both because of the complexity of the systems involved and the difficulty (near impossibility) in controlling all of the variables. One approach which may be taken is the use of artificial models of the processes of interest – of particular interest here are artificial agents, for example real or simulated robots, which may be used as a solution to the aforementioned problems. This very brief discussion looks at the potential benefits of using artificial agents in the examination of biological processes – for example animal behaviour or cognitive abilities.

Firstly, whereas the isolation of a specific process-of-importance is impossible in the natural systems of interest, it is very possible in an artificial system. Indeed, it would be extremely difficult to implement all of the aspects of the biological system in an artificial one. Of course, care must be taken to ensure that the process or function in question remains in context, so that its examination may yield results applicable to the original system. Secondly, in testing humans and animals, it is very difficult to control all of the variables, both experimental and environmental. With artificial agents, every aspect of their functioning is under full control, leading to the easier control of experimental variables. Also, in simulated environments at least, it is also possible to control every aspect of the environment. A final aspect is also of importance, which may be seen as both an advantage and a disadvantage – that of the practicalities of actual implementation. This forces one to be explicit in all the details of a particular theory, to a level which is not normally required from a theory derived from observations of the original biological process. When models of the processes of interest are actually implemented, a number of assumptions are generally required over and above those central to the theory. This forced explicitness may detract from the biological plausibility of the implementation, however it may provide a handle on an alternative perspective on the original biological theory.

Wednesday, February 07, 2007

The Morris Water Maze: Applications to Cognitive Robotics work?

In this post, I will be looking at the Morris Water maze – a rodent testing environment designed to assess spatial, long term and short term (working) memory. After a brief overview, taking into account some considerations when using rodents as test subjects, I will make a few very brief notes on the potential relevance of this to work with artificial cognitive architectures, particularly when embodied (in mobile robots for example). My intention is that a future post will expand on this second part.

The Morris water maze, otherwise known as the water task or maze, is a spatial task which uses simple visual cues [1], and was originally conceived to allow comparisons to be made between spatial memory and classical and instrumental conditioning, although its conceiver later characterised methods specifically for the study of spatial working memory [2]. In this maze, a swimming rodent tries to find a hidden platform in a pool of water, on the basis that most rodents have an aversion to swimming and hence would view the platform as an escape, and thus provide positive reinforcement [3]. Around the edge of the pool are a series of static visual cues by which the rodent can determine direction. Please see the figure for the typical layout of a Morris water maze.


As shown, the platform is visible above the surface of the water – a case that is used for initial training of the rodents – although in most testing situations, this platform is submerged and hence out of sight, as the water is usually made opaque (using milk powder for example). The visual cues may be of two types; normal elements of the room in which the experiments are conducted which are visible from inside the tank (for example a desk or cupboard) which have three dimensions, or as shown the picture, with high visibility two dimensional cues used, and all others excluded. The behaviour of the rodent whilst in the maze is often recorded remotely using a video camera so as not to interfere with the trials. There are few variations of the basic test, although as mentioned, the type of visual cue used and their number and arrangement allow innumerable permutations.


A number of indices are used to assess performance of the rat, the most common of which are: time to reach the platform, path length (and hence swimming speed), and amount of time spent in predetermined areas of the tank [4]. These indices may be significantly influenced by a wide range of environmental and species-specific factors which do not reflect the cognitive abilities of the rodent, such as the sex of the rodent, dimensions of the tank used, water temperature, nutritional status, hormonal status and home cage environment, that must be taken into account [4], but which may actually be inadequately considered [5, 6].


The interpretation of the obtained results with regard the cognitive functions of the rodents in question is often fraught with ambiguities. Taking as an example the measurement of the amount of time spent in a certain area of the tank [4], it is commonly assumed that long periods of time spent in the target sector is indicative of a good memory, as it is supposed to show that the rodent is searching for the platform in approximately the right place. However, if the platform that was present were to be moved, the persistence of the rodent in that sector may be indicative of a reduced cognitive performance, compared to one that quickly ‘gives up’, and moves to another sector for searching there. It is due to difficulties of interpretation such as these that the underlying factors discussed previously must also be taken into account to keep each set of experiments in the correct context, as differences in behaviour may be due to environmental differences rather than cognitive differences.


As discussed, the water maze may be used to examine a wide range of cognitive processes, ranging from spatial memory to navigation. When one is looking at embodied cognitive architectures, one is generally trying to recreate in some form the behaviour of a biological animal due to their obvious ability to cope with an uncertain world – a desired property for any artificial agent. The water maze thus seems to be ideally suited to their behavioural analysis, when suitably modified, as it provides a bounded domain with easily controllable factors, and yet provides a complex problem: that of integrating spatial memory with suitable motor responses after having to learn both the generalities of the type of the domain, and the specific instantiation of the maze in any given trial. A number of robotics studies have in fact already used this idea – these are to be reviewed in a later post.

References:
[1] R. G. M. Morris, "Spatial localization does not require the presence of local cues," Learning and Motivation, vol. 12, pp. 239-260, 1981.
[2] R. Morris, "Developments of a water-maze procedure for studying spatial learning in the rat," Journal of Neuroscience Methods, vol. 11, pp. 47-60, 1984.
[3] G. B. Mulder and K. Pritchett, "The Morris Water Maze," Contemporary Topics in Laboratory Animal Science, vol. 42, pp. 49-50, 2003.
[4] S. L. Allen, "The Morris Water Maze as a tool for Cognitive Function Testing," Syngenta CTL, Macclesfield, Technical Report July 2002 2002.
[5] M. Sarter, "Animal cognition: defining the issues," Neuroscience and Behavioral Reviews, vol. 28, pp. 645-650, 2004.
[6] T. Steckler and J. L. Muir, "Measurement of cognitive function: relating rodent performance with human minds," Cognitive Brain Research, vol. 3, pp. 299-308, 1996.

Tuesday, February 06, 2007

Brain Based Artificial Agents

Brief notes on "Brain-Based Devices: Intelligent systems based on principles of the nervous system", Krichmar and Edelman (2003), Proceedings of the 2003 IEEE/RSJ Int. Conf. on Intelligent Robots and Systems, p940-945.

Krichmar et al have, for a number of years now, been working on the creation of physical devices (i.e. mobile robots) which are controlled by simulated nervous systems - hence the name brain based devices. The Darwin series of robots are the embodiment of this work, and this paper describes Darwin VIII in particular (I believe that Darwin X is the latest incarnation). The intention of this summary, however, is to look more at the principles involved rather than look specifically at the Darwin VIII device.

Brain based devices are constrained by four basic design principles in accordance with the selectional principles by which natural systems develop and operate:

- The device must engage in a behavioural task
- The device's behaviour must be controlled by a simulated nervous system which reflects the brains structure and dynamics
- The devices behaviour is modified through a reward system which indicates the relevance of sensory information to the simulated nervous system
- The device must be situated in the real world

The fourth point explains the need for mobile robotics: they provide the only practical means of 'embodying' the artificial nervous system in the real world. Given yesterdays discussion, the third point is of particular interest. The need for a value signal of some sort is a central part of the behaviour that Darwin VIII learns. The desired behaviour for this device was to learn how to categorise perceptual stimuli without explicit teaching - its task was to 'taste' blocks with two different patterns on top of them by moving around an arena and gripping them. The two different patterned blocks had different electrical conductivities - high conductivity which 'tasted good', and a low conductivity which 'tasted bad' (i.e. the former elicited a positive response from the value system, and the latter a negative response). The task thus required not only recognition, but also the correct motor controls in order to perform the physical separation of the blocks.

The artificial nervous system has six major elements which make up the simulated brain: an auditory system, a visual system, a 'taste' system, sets of motor neurons, a visual tracking system, and a value system. Please refer to the paper for further details. There were a number of behaviours set a priori, however, the choice of which to use in a given situation is performed by the artificial nervous system. Using this architecture, associations were learned between the patterns on the blocks and their 'tastes'. In addition to this, second-order conditioning experiments could be carried out with different stimuli - associations could be learned between different sensory modalities aswell. One of the more interesting results of this work however was the 'emergence' of perceptual categories - i.e. the ability of the artificial nervous system to categorise the stimuli encountered in its environment.

Following on from emergence of perceptual categories, the discussion turned to the binding problem. The Darwin VIII architecture, in creating these categories, was able to solve this problem, by bringing together disparate pieces of perceptual information, to enable the distinction of objects from the background. It was found that the neural circuits responsible for each of the modalities fired synchronously when an object was detected. Different objects were distinguishable in the differing temporal characteristics of these firing patterns.

As a side issue, the artificial nervous system was composed of nearly 20,000 neural units in the six regions described previously. The initial synaptic weights were initially randomly assigned. This allowed the study of what were essentially different 'individuals' with the same architecture, but different initial conditions. These individuals never displayed the same neural activity, even though the architecture was the same, and the overall behaviours displayed were similar. This may have interesting implications when applied to studies of individual differences in humans.

In the final discussion of the paper, Occam's razor is looked at in terms of the development of cognitive architectures: often, the view is taken that the simplest explanation is the best one by modellers. However, in neuroscience this is rarely the case due to the vast complexity of the brain and the myriad of interconnections and cross-influences that need to be taken into account for a full 'picture' to be developed. A number of levels of organisation can be looked at (from the synaptic to the organism), the examination of each is with merit, however, for the full picture, these must be integrated. Thi is where the authors say that brain based devices come in useful. In addition to this role in studying the human brain, the techniques may of course also apply to the construction of devices and methods for industrial and commercial applications. All based on neurobiological rather than computational principles of construction.

Monday, February 05, 2007

Emotion and Decision Making

Notes on "Emotion, Decision making and the Orbitofrontal Cortex", A. Bechara, H. Damasio, A. Damasio (2000), Cerebral Cortex, vol 10, p295-307.

The Somatic marker hypothesis (defined by Damasio, 1994 and 1996) says that a defect in emotion and feeling has a detrimental effect on decision making - it also proposes a number of brain structures thought to underlie this effect. Emotions in this theory are defined to be 'somatic states' as they are said to be primarily represented in the brain by "transient changes in the activity pattern of somatosensory structures" ('somatic' essentially refers to the internal environment). This paper looks at this theory, focusing particularly on the role of the orbitofrontal cortex and its interaction with the emotion regions of the brain, in addition to a discussion of the relationship between these two distinct functions (decision making and emotion) and the cognitive function of working memory.

There are four basic assumptions upon which the somatic marker theory is based, which are detailed at the start of the paper. (1) many levels of neural operations are responsible for human reasoning and decision making; (2) all cognitive processes depend upon other, supporting, processes such as attention, working memory and emotion; (3) any reasoning and decision making that occurs relies on the availability of information relevant to the current situation - this information is stored in a 'dispositional' form (i.e. implicit knowledge); (4) knowledge may be categorised in one of four ways: "(a) innate and acquired knowledge concerning bioregulatory processes and body states and action, including those which are mad explicit as emotions; (b) knowledge about entities, facts (e.g. relations, rules), actions and action complexes (stories), which are usually made explicit as images; (c) knowledge about the linkages between (a) and (b) items, as reflected in individual experience; (d) knowledge resulting from the categorisation of items in (a), (b), and (c)." On a side note, point (b) implies the presence of at least pseudo-symbols – a prevalent idea in the cognitive sciences, but one which has been strongly debated.

It is upon these four assumptions that the somatic marker hypothesis is further detailed. The orbitofrontal cortex is recognised to be an essential part of the decision making structures – its supposed function in this regard is to learn associations between the present situation on the one hand (or certain aspects of the sensory situation) and emotional (or other bioregulatory) states on the other. This provides a link between the individuals past emotional experience and the present need to make a decision. The somatic marker hypothesis thus proposes that this central linking of emotion to past experiences reduces the decision making space to enable efficient decision making among those options still available after the ‘pre-filtering’ by emotional processes. It is this process, according to the authors, that allows an animal to make an efficient decision on a short time scale. The rest of the article provides an overview of the evidence supporting this hypothesis, of which I will review briefly only a few points.

Using a gambling task, the Skin Conductance Responses (SCRs) of the subjects was taken during performance. The gambling task involved four decks of cards (A, B, C and D), with associated rewards and punishments (in the form of money), arranged such that the short term rewards from two of the decks of cards (A and B) were high in the short term compared to the other two (decks C and D). However, the second set of two decks of cards had a low associated overall loss, whereas the first two decks had a high loss. Overall, in the long-term, it was better to pick cards from decks C and D. The subjects aim was naturally to make as much money as possible. Four main points arose during these studies. Firstly, patients with ventro-medial prefrontal cortex (VM patients) damage exhibited impairment compared to control subjects, which remained constant over time – which is concurrent with the observation in their everyday lives that they are unable to learn from their previous mistakes. Secondly, biases guide decisions. SCRs of participants showed that a ‘sub-conscious’ and emotion-driven prediction of reward or punishment bias forms which does have an effect on the decisions taken, even prior to the conscious awareness of this. Thirdly, there was a suggestion that risk-taking behaviour, which has in the past been linked to prefrontal cortex damage, is not synonymous with impaired decision making (a dissociation between the two). Finally, and following on from the second point, evidence was presented which showed that the biases do not have to be conscious to strongly influence decision making, as demonstrated by the SCR results.

Based on this preliminary evidence in support of the somatic marker hypothesis, a discussion is presented on the relationship between emotion, memory and decision making – two main points caught my eye. It is known that the memory of facts can be enhanced when learned in association with an emotion – from the point of view of the preceding discussion, a natural question to ask would be is working memory, and other memory processes, distinct from those involved in decision making? The answer, in short, is essentially yes: there is a double dissociation (anatomic and cognitive) between decision making on the one hand (anterior VM region), and working memory on the other (right side dorsal lateral prefrontal cortex). A second pertinent question would be whether the emotion mechanism responsible for the increase in memory capabilities is different from the one which influences decision making. The amygdala is known to be central to the workings of emotional processing, and the question thus becomes whether the processing of the two aforementioned processes be distinguished in the amygdala. The answer again seems to yes: experimental results indicate that VM patients are capable of enhancing working memory with emotional cues/responses, but that decision making is impaired, as discussed previously. The final section of the paper describes preliminary work towards answering some of the questions which have arisen during the research, such as "Why do VM patients fail to generate these biases or emotional signals?". Essentially, the "nature of the mechanism responsible for the failure of VM patients to trigger somatic states when pondering decisions remain unspecified."

Most current theories on decision making postulate that decisions are made on the basis of an assessment of the future benefit of possible future outcomes of executing a particular action – i.e. ‘cold, hard logic’ (my words). Emotion may be listed as a contributing and influencing factor, but it is never central as in the somatic marker hypothesis. To recap, this hypothesis "proposes that individuals make judgements not only by assessing the severity of outcomes and their probability of occurrence, but also and primarily in terms of their emotional quality." The few other theories in which emotion takes a central role in decision making (e.g. that by Rolls) essentially view the body as introducing noise into the decision making processes – the somatic marker hypothesis takes an alternate view that these body signals (bioregulatory signals, including emotions) are essential to the decision making process – the reducing of the decision space as mentioned previously.

The implications of this work for artificial cognitive architectures, and thus also in cognitive robotics, are obvious. The view that emotion is something which is central to decision making, and not just an ‘optional extra’ which may add a little interesting functionality (as it may be seen by some), seems to be put to rest by this work on emotion and decision making in humans. So instead of tacking emotion-like constructs onto already fully formed cognitive architectures, it seems necessary to take this as an initial consideration. The point concerning the reducing of the decision space is a particularly interesting one (as previously raised): could this be a small step towards providing a solution to the frame problem for artificial intelligence, and cognitive robotics?

A reference in the paper to another paper on the influence of emotion on memory, which may be of interest:
Cahill et al. (1995), "The Amygdala and Emotional Memory", Nature, 388, pp295-296.

Sunday, February 04, 2007

The Week of Science

The week of science starts tomorrow. I intend to use this as an index for the posts which will appear over the week, so will update it daily. I'm looking forward to the challenge - hope I can stick with it.

Day 1: Emotion and Decision Making
Day 2: Brain based Artificial Agents
Day 3: The Morris Water Maze: Applications to Cognitive Robotics work?
Day 4: A brief discussion of the justification of the use of Artificial Agents in studying Biological Processes
Day 5: Biological Gyroscopes
Day 6: An Overview of Humanoid Robotics
Day 7: A Review of Just Science Week

The Week of Science Feed can be found here.

Friday, February 02, 2007

How to report scientific research to a general audience

Would just like to point out a great post on Cognitive Daily.

Written by Dave Munger, it's an 11 point guide to reviewing technical journal papers for a general audience, though I reckon very similar principles should apply even if the audience is specialised in the field. I think everyone tries to go by these guidelines in some shape or form, but it's always useful to have these things spelled out explicitly - even if it's just used as a casual reminder every once in a while.

Link to post.

Thursday, February 01, 2007

Philosophia Naturalis #6

Philosophia Naturalis #6 has been published at Science and Reason!

Most of the posts in this issue seem to be on Physics and Astronomy, and there are a few on mathematical concepts that arise nearly everywhere (for example Fourier transforms and Bayes theorem).