We introduce a mixed integer program (MIP) for assigning importance scores to each neuron in deep neural network architectures which is guided by the impact of their simultaneous pruning on main learning task network. By carefully devising objective function MIP, we drive solver minimize number critical neurons (i.e., with high score) that need be kept maintaining overall accuracy trained Furth...