Comprehensive Working Memory Activation in SOAR

Notes on Nuxoll, Laird and James (2004), "Comprehensive working memory activation in SOAR", Proc. of the 6th Int. Conf.on Cognitive Modelling, pp 226-230

The SOAR cognitive architecture uses exclusively symbolic representation and processing. A recent addition to the basic working memory/procedural memory components of the SOAR architecture is an episodic memory component. An essential part of the functioning of this is the retrieval of the most relevant episode. This is a non-trivial problem due to the number of irrelevances in the episode. Biasing potential matches is thus a potential solution: this paper covers the possibility of using activation as a biasing factor.

Working memory in SOAR is made up of a collection of attribute-value pairs, such as the current state (including external sensory information and internal inferences). If the contents of working memory matches the condition of any production rule in procedural memory (of the form condition->action), then that rule fires (or is executed) - even if multiple rules are activated. Each rule has the capability of modifying the contents of working memory, resulting in a possible action, or the firing of another rule. The characteristics of the SOAR architecture which are relevant to this particular study are as follows: (1) simultaneous rule firing (in the case where the contents of working memory match the conditions of multiple productions); (2) operators (in the case where multiple rules fire, they each propose an operator. Only one of these operators can be selected for application to the contents of working memory); (3) the decision cycle (propose, select and apply); (4) persistent working memory (two types: o-supported indicating that the contents of working memory remain intact until explicitly modified or cleared, and i-supported which are cleared from working memory when the rule which instantiated the particular element ceases to fire, i.e. match working memory).

The original study to employ activation levels in the SOAR architecture was Chong (2003). This study proposed that part of the contents of working memory were subject to the effect of activation levels, where an item in working memory with a low activation would eventually be removed, even if it was an o-supported working memory element. The current paper under review changed this to apply to the entire contents of working memory (apart from i-supported, which is automatically removed anyway), although they kept the underlying principle of activation alteration: the more an element is used, the higher its activation level, and vice versa for litle used elements (ln decay used). The activation assigned to a newly created production is based upon the activations of those elements which contribute to its creation - this is a different approach to those methods which automatically assign newly created rules a relatively high initial activation level. One final general point on activation levels are that each reference to an element in working memory has its own activation level, with the overall element activation being a sum of each reference. Thus, each reference to an element can decay independently. For further details please refer to the paper.

This updated version of SOAR is then tested in a 16x16 gridworld, in which the SOAR architecture controls the movements of an agent (or 'eater'), in order to examine the role activation has in aiding learning, specifically in retrieving past episodic experiences, as mentioned previously. In this case, retrieved past experiences were used as a comparison to the action the agent was currently considering, thus illustrating the importance of selecting the 'correct' past experience. When activation was enabled, the eater showed an approximate 30% improvement in performance over an agent which did not use activation to select an episode.

Because of the fact that i-supported working memory elements do not have an activation level, this leads to the possibility of i-support masking, which occurs when the o-supported elements supporting an i-element are removed, thus removing the i-element, even though the i-element may have been referenced (refer to figure 5). To rectify this problem, a 'pay it backward' system was implemented, whereby activation levels of supporting o-elements may be increased if a dependant i-element is referenced. At this stage also, the 'pay it forward' scheme of basing the activation level of newly created productions on the activations of those productions from which it was created was also introduced. The overall results of experimentation (still using the eater simulation scheme) were clear. The pay-it-backward addition improved performance over the activation alone, and the addition of the pay-it-forward scheme showed still further improvement.