Networks are not 'hidden rules'

The grammatical morphology of Hebrew-speaking children with specific language impairment sensory and language processing in language-Does learning a language involve formulating rules or gathering statistics? Marcus and colleagues offered two pieces of evidence bearing on this debate 1. Their behavioral studies were taken as evidence that babies form 'alge-braic rules' and their attempt to model this behavior suggested that it is incompatible with the properties of a 'popular' class of connectionist networks. Both of these claims have been the subject of considerable discussion elsewhere 2–9. In this letter we would like to raise a more general issue about the relationship between connectionist models and algebraic rules. More specifically, we would like to examine critically the statement by Marcus that 'Seidenberg and Elman have not gotten rid of the rule; they have simply hidden it', in reference to simulation data that we recently reported 2,5,7. The purpose of our simulation was to demonstrate that the sequential regularities implicit in Marcus et al.'s stimuli provided a sufficient basis for differentiating sequences that conformed to the 'algebraic rule' from ones that did not, and that having acquired this information , a network could generalize appropriately to novel stimuli. The model was not an account of exactly how babies acquire this information; rather, it demonstrated that if they encoded this information, by whatever means, it would provide a basis for the observed behavior. That babies are able to detect such regularities is consistent with an extensive empirical literature 10. Marcus objected to two aspects of this simulation: instead of the prediction task used in some other models (including Marcus et al.'s failed simulation), our model was trained to categorize stimuli as fitting a pattern or not doing so. We also used a supervised-learning procedure in which the network was provided with explicit feedback. In a sense, then, feedback to the model was structured on the basis of a rule; hence, Marcus concluded that the model must have had the rule 'hidden' in it. However, his statement is a non sequitur. Merely training a network to categorize stimuli into two groups using explicit feedback does not cause it to formulate a rule. Categorization, like prediction, is a task. The theoretical issue is how such tasks are performed. For a while now people have been debating two competing accounts. One is that such tasks involve formulating rules that have specific properties: for example, they operate over …