Neural Decision Trees
نویسنده
چکیده
In this paper we propose a synergistic melting of neural networks and decision trees (DT) we call neural decision trees (NDT). NDT is an architecture a la decision tree where each splitting node is an independent multilayer perceptron allowing oblique decision functions or arbritrary nonlinear decision function if more than one layer is used. This way, each MLP can be seen as a node of the tree. We then show that with the weight sharing asumption among those units, we end up with a Hashing Neural Network (HNN) which is a multilayer perceptron with sigmoid activation function for the last layer as opposed to the standard softmax. The output units then jointly represent the probability to be in a particular region. The proposed framework allows for global optimization as opposed to greedy in DT and differentiability w.r.t. all parameters and the input, allowing easy integration in any learnable pipeline, for example after CNNs for computer vision tasks. We also demonstrate the modeling power of HNN allowing to learn union of disjoint regions for final clustering or classification making it more general and powerful than standard softmax MLP requiring linear separability thus reducing the need on the inner layer to perform complex data transformations. We finally show experiments for supervised, semi-suppervised and unsupervised tasks and compare results with standard DTs and MLPs.
منابع مشابه
Predicting The Type of Malaria Using Classification and Regression Decision Trees
Predicting The Type of Malaria Using Classification and Regression Decision Trees Maryam Ashoori1 *, Fatemeh Hamzavi2 1School of Technical and Engineering, Higher Educational Complex of Saravan, Saravan, Iran 2School of Agriculture, Higher Educational Complex of Saravan, Saravan, Iran Abstract Background: Malaria is an infectious disease infecting 200 - 300 million people annually. Environme...
متن کاملEstimating Suspended Sediment by Artificial Neural Network (ANN), Decision Trees (DT) and Sediment Rating Curve (SRC) Models (Case study: Lorestan Province, Iran)
The aim of this study was to estimate suspended sediment by the ANN model, DT with CART algorithm and different types of SRC, in ten stations from the Lorestan Province of Iran. The results showed that the accuracy of ANN with Levenberg-Marquardt back propagation algorithm is more than the two other models, especially in high discharges. Comparison of different intervals in models showed that r...
متن کاملTagging heavy flavours with boosted decision trees
This paper evaluates the performance of boosted decision trees for tagging b-jets. It is shown, using a Monte Carlo simulation of WH → lνqq̄ events that boosted decision trees outperform feed-forward neural networks. The results show that for a b-tagging efficiency of 90% the b-jet purity given by boosted decision trees is almost 20% higher than that given by neural networks.
متن کاملKnowledge Extraction From Trained Neural Networks
Received Jul 16 th , 2012 Revised Aug 01 th , 2012 Accepted Sept 02 th , 2012 Artificial neural networks (ANN) are very efficient in solving various kinds of problems But Lack of explanation capability (Black box nature of Neural Networks) is one of the most important reasons why artificial neural networks do not get necessary interest in some parts of industry. In this work artificial neural n...
متن کاملA multivariate approach to heavy flavour tagging with cascade training
This paper compares the performance of artificial neural networks and boosted decision trees, with and without cascade training, for tagging b-jets in a collider experiment. It is shown, using a Monte Carlo simulation of WH → lνqq̄ events, that boosted decision trees outperform artificial neural networks. Furthermore, cascade training can substantially improve the performance of both boosted dec...
متن کاملFuzzy min-max neural network based decision trees
This paper presents a new decision tree learning algorithm, fuzzy min-max decision tree (FMMDT) based on fuzzy min-max neural networks. In contrast with traditional decision trees in which a single attribute is selected as the splitting test, the internal nodes of the proposed algorithm contain a fuzzy min-max neural network. In the proposed learning algorithm, the exibility inherent in the fuz...
متن کاملذخیره در منابع من
با ذخیره ی این منبع در منابع من، دسترسی به آن را برای استفاده های بعدی آسان تر کنید
برای دانلود متن کامل این مقاله و بیش از 32 میلیون مقاله دیگر ابتدا ثبت نام کنید
ثبت ناماگر عضو سایت هستید لطفا وارد حساب کاربری خود شوید
ورودعنوان ژورنال:
- CoRR
دوره abs/1702.07360 شماره
صفحات -
تاریخ انتشار 2017