Toward Robust Uncertainty Estimation with Random Activation Functions

Deep neural networks are in the limelight of machine learning with their excellent performance many data-driven applications. However, they can lead to inaccurate predictions when queried out-of-distribution data points, which have detrimental effects especially sensitive domains, such as healthcare and transportation, where erroneous be very costly and/or dangerous. Subsequently, quantifying uncertainty output a network is often leveraged evaluate confidence its predictions, ensemble models proved effective measuring by utilizing variance over pool models. In this paper, we propose novel approach for quantification via ensembles, called Random Activation Functions (RAFs) Ensemble, that aims at improving diversity toward more robust estimation, accommodating each different (random) activation function. Extensive empirical study demonstrates RAFs Ensemble outperforms state-of-the-art methods on both synthetic real-world datasets series regression tasks.