A new method for inferring specific stochastic grammars is presented. The process called Hybrid Model Learner (HML) applies entropy rate to guide the agglomeration process of type ab->c. Each rule derived from the input sequence is associated with a certain entropy-rate difference. A grammar automatically inferred from an example sequence can be used to detect and recognize similar structures i...

Information-theoretic methods have been widely used in neuroscience, in the broad effort to analyze and understand the fundamental informationprocessing tasks performed by the brain. In these studies, the entropy has been adopted as the main measure for quantifying the amount of information transmitted between neurons, via the spike trains they generate. One of the first and most important goal...

We derive the relative entropy between two Markov transition rate matrices from sample path considerations. This relative entropy is interpreted as a \level 2.5" large deviations action functional. That is, the level two large deviations action functional for empirical distributions of continuous-time Markov chains can be derived from the relative entropy using the contraction mapping principle...

We investigate quantum dynamical systems defined on a finite-dimensional Hilbert space and subjected to an interaction with an environment. The rate of decoherence of initially pure states, measured by the increase of their von Neumann entropy, averaged over an ensemble of random pure states, is proved to be bounded from above by the partial entropy used to define the ALF-dynamical entropy. The...

Abstract. For a stationary Markov process the detailed balance condition is equivalent to the timereversibility of the process. For stochastic differential equations (SDE’s), the time discretization of numerical schemes usually destroys the time-reversibility property. Despite an extensive literature on the numerical analysis for SDE’s, their stability properties, strong and/or weak error estim...

This paper is concerned with the connection between two classes of population variables: measures of population growth rate--the Malthusian parameter, the net reproduction rate, the gross reproduction rate, and the mean life expectancy; and measures of demographic heterogeneity--population entropy. It is shown that the entropy functions predict the response of the growth rate parameters to pert...

We develop ideas proposed by Van der Straeten to extend maximum entropy principles to Markov chains. We focus in particular on the convergence of such estimates in order to explain how our approach makes possible the estimation of transition probabilities when only short samples are available, which opens the way to applications to non-stationary processes. The current work complements an earli...

The entropy production rate was determined for avascular tumor growth. The proposed formula relates the fractal dimension of the tumor contour with the quotient between mitosis and apoptosis rate, which can be used to characterize the degree of proliferation of tumor cells. The entropy production rate was determined for fourteen tumor cell lines as a physical function of cancer robustness. The ...

The entropy constancy principle describes the tendency for information in language to be conveyed at a constant rate. We explore the possible role of this principle in spoken dialog, using the “summed entropy rate,” that is, the sum of the entropies of the words of both speakers per second of time. Using the Switchboard corpus of casual dialogs and a standard ngram language model to estimate en...