The Properties of Higher Cognitive Processes and How They Can be Modelled in Neural Nets

It is proposed that the distinction between basic and higher cognitive processes can be captured by the difference between associative and relational processes. Properties of relational processing include reification of the link between entities, so higher-order relations have lower-order relations as arguments, whereas an associative link per se cannot be a component of another association. Therefore relational processes can be hierarchical and recursive, whereas associative structures are flat. Relations, unlike associations, also have the properties of omni-directional access and systematicity. Relational processes support reasoning and contentindependent transfer, and have many of the properties of symbolic models. Typical feedforward neural nets do not implement these properties in a natural way, but they can be implemented with tensor product nets. The requirements for neural nets to model higher cognitive processes are considered. The debate about the nature of higher cognition has included claims that it is symbolic, compositional and systematic, and that it cannot be modelled by associative architectures (Fodor & Pylyshyn, 1988). On the other hand it has often been shown that human reasoning does not readily conform to logic, and is better modelled by more content-specific processes such as mental models, that function as analogues (Johnson-Laird & Byrne, 1991; Niklasson & van Gelder, 1994). There has been a parallel debate about whether learning is conscious or unconscious, and whether what is learned is rule-based or instance-based (Shanks & St. John, 1994). These dichotomies also overlap to some extent with the implicit-explicit distinction (Clark & Karmiloff-Smith, 1993). As Hadley (1994) points out, we still lack a clear definition of the nature of higher cognition and, consequently, we are unable to clearly distinguish it from more basic processes. One consequence of this situation is that the criteria that neural net models of higher cognition need to fulfil have not been defined. In this paper we propose that higher cognitive processes entail representing and processing explicit relations, whereas basic processes can be identified with associations. This in turn constrains the type of neural net models that are required. Higher cognitive processes depend on a set of structure-sensitive rules for operating on representations. In language syntactic rules define relations between words (e.g. parts of speech or case roles), but reasoning also depends on rules that relate entities, partly independent of content; e.g., the idea that fruit includes apples and non-apples is an instance of complementation or (in the Psychological literature) inclusion, but we can also understand inclusion as nonempty sets a and a’ being included in b, independent of specific instances. A structure is a set on which one or more relations is defined. Because relations are the essence of structure, the structural properties of higher cognitive processes can be captured by systems that process relations. However, despite its importance, and intensive study in disciplines such as computer science (Codd, 1990) the theory of relational knowledge has received little attention in Psychology (Smith, 1989).