Exponentially many local minima for single neurons
نویسندگان
چکیده
We show that for a single neuron with the logistic function as the transfer function the number of local minima of the error function based on the square loss can grow exponentially in the dimension.
منابع مشابه
Exponentially Vanishing Sub-optimal Local Minima in Multilayer Neural Networks
Background: Statistical mechanics results (Dauphin et al. (2014); Choromanska et al. (2015)) suggest that local minima with high error are exponentially rare in high dimensions. However, to prove low error guarantees for Multilayer Neural Networks (MNNs), previous works so far required either a heavily modified MNN model or training method, strong assumptions on the labels (e.g., “near” linear ...
متن کاملTrivializing The Energy Landscape Of Deep Networks
We study a theoretical model that connects deep learning to finding the ground state of the Hamiltonian of a spherical spin glass. Existing results motivated from statistical physics show that deep networks have a highly non-convex energy landscape with exponentially many local minima and energy barriers beyond which gradient descent algorithms cannot make progress. We leverage a technique know...
متن کاملAdiabatic optimization without local minima
Several previous works have investigated the circumstances under which quantum adiabatic optimization algorithms can tunnel out of local energy minima that trap simulated annealing or other classical local search algorithms. Here we investigate the even more basic question of whether adiabatic optimization algorithms always succeed in polynomial time for trivial optimization problems in which t...
متن کاملMinimal potentials with very many minima
We demonstrate, by construction, that simple renormalizable matrix potentials with S(N), as opposed to O(N), symmetry can exhibit an exponentially large number of inequivalent deep local minima.
متن کاملLifetime of Stringy de Sitter Vacua
In this note we perform a synopsis of the life-times from vacuum decay of several de Sitter vacuum constructions in string/M-theory which have a single dS minimum arising from lifting a pre-existing AdS extremum and no other local minima existent after lifting. For these vacua the decay proceeds via a Coleman–De Luccia instanton towards the universal Minkowski minimum at infinite volume. This c...
متن کاملذخیره در منابع من
با ذخیره ی این منبع در منابع من، دسترسی به آن را برای استفاده های بعدی آسان تر کنید
عنوان ژورنال:
دوره شماره
صفحات -
تاریخ انتشار 1995