This paper describes a machine learning method, called Regression by Selecthtg Best P~’ttllll’es (RSBF). RSBF consists of two phases: The first phase aims to find the predictive power of each feature by constructing simple linear regression lines, one per each continuous feature and number of categories pen each categorical feature. Although the predictive power of a continuous feature is const...

We study a regression problem where for some part of the data we observe both label variable (Y) and predictors (X), while other only are given. Such arises, example, when observations costly may require skilled human agent. When conditional expectation E[Y|X] is not exactly linear, one can consider best linear approximation to expectation, which be estimated consistently by least-square estima...

The Perceptron algorithm elegantly solves binary classification problems that have a margin between positive and negative examples. Isotonic regression (fitting an arbitrary increasing function in one dimension) is also a natural problem with a simple solution. By combining the two, we get a new but very simple algorithm with strong guarantees. Our ISOTRON algorithm provably learns Single Index...