Results of a simulation study of the fit of data to an estimated parametric model are reported. Three particular models including the two-parameter normal and exponential distributions, and the simple linear regression model are considered. A number of scaled versions of the least squares residuals from the regression model, and quantities that may be called residuals from the other two models ...

We consider planning in a Markovian decision problem, i.e., the problem of finding a good policy given access to a generative model of the environment. We propose to use fitted Q-iteration with penalized (or regularized) least-squares regression as the regression subroutine to address the problem of controlling model-complexity. The algorithm is presented in detail for the case when the functio...

In this paper we develop a theoretical analysis of the performance of sampling-based fitted value iteration (FVI) to solve infinite state-space, discounted-reward Markovian decision processes (MDPs) under the assumption that a generative model of the environment is available. Our main results come in the form of finite-time bounds on the performance of two versions of sampling-based FVI. The co...

Raytracing metaballs is a problem that has numerous applications in the rendering of dynamic soft objects such as fluids. However, current techniques are either limited in the visual effects that they can render or their performance drops as the number of metaballs and their density increase. We present a new acceleration structure based on BVH and kd-tree for efficient raytracing of a large nu...

Fitted Q-Iteration (FQI) is a popular approximate value iteration (AVI) approach that makes effective use of off-policy data. FQI uses a 1-step return value update which does not exploit the sequential nature of trajectory data. Complex returns (weighted averages of the n-step returns) use trajectory data more effectively, but have not been used in an AVI context because of off-policy bias. In ...

Compound action potential (CAP) of spinal cord represents valuable properties of neural fibers including excitability, rate of myelination and membrane integrity. These properties are measured using amplitude, latency and area under curve of CAPs recorded from spinal cord. Here, the isolated spinal cord was set in a double sucrose gap (DSG) chamber and its response to intracellular stimulation ...