Massively-parallel connectionist networks have traditionally been applied to constraint-satisfaction in early visual processing (Ballard, Hinton & Sejnowski, 1983), but are now being applied to problems ranging from the TravelingSalesman Problem to language acquisition (Rumelhart & MeClelland, 1986). In these networks, knowledge is represented by the distributed pat tern of activity in a large ...

This paper describes a new parallel Logic Programming language designed to exploit the ORand Independent ANDparallelisms. The language is based on conventional Prolog but with natural extensions to support handling of multiple solutions and expression of parallelism. 1 . I n t r o d u c t i o n PEPSys (Parallel ECRC Prolog System) is a research project started in mid 1984 in the Computer Archit...

We introduce a new parallel programming paradigm namely synchronous parallel critical sections Such parallel critical sections must be seen in the context of switching between synchronous and asynchronous modes of computation Thread farming allows to gen erate bunches of threads to solve independent subprob lems asynchronously and in parallel Opposed to that synchronous parallel critical sectio...

Features of an explicitly parallel programming language targeted for reconfigurable parallel processing systems, where the machine's -1processing elements (PE's) are capable of operating in both the SIMD and SPMD modes of parallelism, are described. The SPMD (Single Program-Multiple Data) mode of parallelism is a subset of the MIMD mode where all processors execute the same program. By providin...

Training neural language models (NLMs) is very time consuming and we need parallelization for system speedup. However, standard training methods have poor scalability across multiple devices (e.g., GPUs) due to the huge time cost required to transmit data for gradient sharing in the backpropagation process. In this paper we present a sampling-based approach to reducing data transmission for bet...

This paper describes SYNAPSIS, a parser for performing real-time understanding of spoken utterances in a parallel computational environment. Understanding continuous speech allowing reasonably free syntax poses two main oroblems, namely the risk of erroneous interpretations and the largeness of the search space owing to the high uncertainty of the input. The parser is characterized by an approa...

The goal of the research described in this paper is to develop flexible language constructs for writing large data parallel numerical programs for distributed memory (MIMD) multiprocessors. Previously, several models have been developed to support synchronization and comnmnication. Models for global synchronization include SIMD (Single Instruction Multiple Data), SPMD (Single Program Multiple D...

We introduce factored language models (FLMs) and generalized parallel backoff (GPB). An FLM represents words as bundles of features (e.g., morphological classes, stems, data-driven clusters, etc.), and induces a probability model covering sequences of bundles rather than just words. GPB extends standard backoff to general conditional probability tables where variables might be heterogeneous typ...

Multigrid algorithms are a computational paradigm that enjoy widespread use in the scientific community. While parallel multigrid applications have been in use for quite some time, parallel language support for features common to multigrid algorithms has been lacking. This forces scientists either to express their computations in high-level terms without knowing the parallel impact, or to expli...

The development of programming languages suitable to express parallel algorithms in is crucial to the pace of acceptance of parallel processors for production applications. As in sequential programming, portability of parallel software is a strongly desirable feature. Portability in this respect means that given an algorithm description in a parallel programming language, it must be possible, w...