Speed Gradient and MaxEnt principle for Shannon and Tsallis entropies

The notion of entropy is widely used in modern statistical physics, thermodynamics, information theory, engineering etc. In 1948, Claude Shannon introduced his information entropy for an absolutely continuous random variable x having probability density function (pdf) p. In 1988, Constantino Tsallis introduced a generalized Shannon entropy. Tsallis entropy have found applications in various scientific fields such as chemistry, biology, medicine, economics, geophysics, etc. A phenomenon when system tends to its state of maximum entropy is known as the maximum entropy (MaxEnt) principle. Since seminal works of E.T. Jaynes (1957) and until recent years the MaxEnt principle attracts a strong interest of researchers. The MaxEnt principle defines the asymptotic behavior of the system, but does not say anything about how the system moves to an asymptotic behavior. Despite a large number of publications studying the maximum entropy states, the dynamics of evolution and transient behavior of the systems are still not well investigated. In this paper we consider dynamics of non-stationary processes that follow the MaxEnt principle. We have derived a set of equations describing dynamics of pdf for Shannon and Tsallis entropies. Systems with discrete probability distribution and continuous pdfs are considered under mass conservation and energy conservation constraints. The uniqueness of the limit pdf and asymptotic convergence of pdf are examined. Convergence of pdfs does not lead to the convergence of the corresponding differential entropies. Based on sufficient conditions the nontrivial convergence of differential entropies is proved. We use the speed-gradient (SG) principle originated in control theory [1]. Applicability of the SG principle has already been experimentally tested for the systems of finite number of particles simulated with the molecular dynamics method [2,3]. We apply similar approach for the systems with discrete and continuous probability distributions considering Shannon and Tsallis entropies. SG principle generates equations for the transient (non-stationary) states of the system operation, i.e. it gives an answer to the question of ”How the system will evolve?” This fact distinguishes the SG principle from MaxEnt principle, the principle of maximum Fisher information and others characterizing the steady-state processes and providing an answer to the questions of ”To where?” and ”How far?”