We analyse the role of the modal axiom corresponding to the first-order formula “∃y (x = y)” in axiomatisations of two-dimensional propositional modal logics. One of the several possible connections between propositional multi-modal logics and classical first-order logic is to consider finite variable fragments of the latter as ‘multi-dimensional’ modal formalisms: First-order variable-assignme...

In this paper we describe some of the research on multi-modal dialogue being performed in the Interactive Systems group at the Navy Center for Applied Research in Artificial Intelligence at the Naval Research Laboratory. Our concern with multi-modal interfaces arises from our past work in developing general-purpose natural language interfaces that can be applied to a variety of decision support...

After perceiving multi-modal behaviour from a user or agent a conversational agent needs to be able to determine what was intended with that behaviour. Contextual variables play an important role in this process. We discuss the concept of context and its role in interpretation, analysing a number of examples. We show how in these cases contextual variables are needed to disambiguate multi-modal...

In recent years human-machine interaction has increased its importance. One approach to an ideal human-machine interaction is develop a multi-modal system behaves like human-beings. This paper introduces an overview on multimodal corpora which are currently developed in Japan for the purpose. The paper describes database of 1)Multi-modal interaction, 2)Audio-visual speech, 3)Spoken dialogue wit...

This paper describes our ongoing work on resolving third person pronouns and deictic words in a multi-modal corpus. We show that about two thirds of these referring expressions have antecedents that are introduced by pointing gestures or by haptic-ostensive actions (actions that involve manipulating an object). After describing our annotation scheme, we discuss the co-reference models we learn ...

In this paper we address the problem of multi-modal coregistration of medical 3D images. Several techniques for the rigid registration of multi-modal images have been developed; in one of those the Kullback-Leibler distance is used to align 2D-3D angiographic images [1]. In this paper we investigate the performance of this technique on the registration of pairs of 3D CT/US images. We study the ...

In recent years, Fully Convolutional Networks (FCN) have led to a great improvement of semantic labeling for various applications including multi-modal remote sensing data. Although different fusion strategies have been reported for multi-modal data, there is no in-depth study of the reasons of performance limits. For example, it is unclear, why an early fusion of multi-modal data in FCN does n...

In this paper, we consider the influence of trust on the assimilation of acquired information into an agent’s belief. By use of modal logic tools, we characterize the relationship among belief, information acquisition and trust both semantically and axiomatically. The belief and information acquisition are respectively represented by KD45 and KD normal modal operators, whereas trust is expresse...

Medical image registration is time-consuming but can be sped up employing parallel processing on the GPU. Normalized mutual information (NMI) is a well performing similarity measure for performing multi-modal registration. We present CUDA based solutions for computing NMI on the GPU and compare the results obtained by rigidly registering multi-modal data sets with a CPU based implementation. Ou...

We describe a multi-modal unscented Kalman filter developed for estimation of aircraft position, velocity and heading from noisy surface surveillance data. The raw data is composed of tracks generated by the Airport Surface Detection Equipment, Model-X at Boston Logan International Airport, and is obtained from the Runway Status Lights system. The multi-modal filter formulation facilitates esti...