Sequential Estimation Of The Steady-State Variance In Discrete Event Simulation

For sequential output data analysis in non-terminating discrete-event simulation, we consider three methods of point and interval estimation of the steady-state variance. We assess their performance in the analysis of the output of queueing simulations by means of experimental coverage analysis. Over a range of models, estimating variances turns out to involve considerably more observations than estimating means. Thus, selecting estimators with good performance characteristics is even more important. INTRODUCTION The output sequence {x} = x1, x2, . . . of a simulation program is usually regarded as the realisation of a stochastic process {X} = X1, X2, . . .. In the case of steady-state simulation, we assume this process to be stationary and ergodic. Current analysis of output data from discrete event simulation focuses almost exclusively on the estimation of mean values. Thus, the literature on “variance estimation” mostly deals with the estimation of the variance of the mean, which is needed to construct a confidence interval of the estimated mean values. In this paper, we are interested in finding point and interval estimates of the steady-state variance σ = Var[Xi] and the variance of the variance, from which we can construct confidence intervals for the variance estimates. Similar to the estimation of mean values, one problem in variance estimation is caused by the fact that output data from steadystate simulation are usually correlated. The variance we estimate is not to be confused with the quantity σ 0 = limn→∞ nVar[X(n)], sometimes referred to as variance parameter (Chen and Sargent, 1990) or steady-state variance constant (Steiger and Wilson, 2001), and which is important in the methods of standardized time series (Schruben, 1983) and various methods using this concept. Applications for the estimators we propose can be found in the performance analysis of communication networks. In audio or video streaming applications, for example, the actual packet delay is less important than the packet delay variation or jitter (see e.g. Tanenbaum, 2003). Other applications include estimation of safety stock or buffer sizes, and statistical process control. Our estimation procedures are designed for sequential estimation. As more observations are generated, the estimates are continually updated, and simulation is stopped upon reaching the required precision. In simulation practice, one observes an initial transient phase of the simulation output due to the initial conditions of the simulation program, which are usually not representative of its long-run behaviour. It is common practice to let the simulation “warm up” before collecting observations for analysis. For many processes, σ converges to its steady-state value slower than the process mean; therefore, existing methods of detection of the initial transient period with regard to the mean value may sometimes not be applicable for variance estimation. A method based on distributions, which includes variance, is described in (Eickhoff et al., 2007). This is, however, not the focus of this paper, so we use a method described in (Pawlikowski, 1990). In the next section we present three different methods of estimating the steady-state variance. We assessed these estimators experimentally in terms of the coverage of confidence intervals. The results of the experiments are presented in Section 3. The final section of the paper summarises our findings and gives an outlook on future research. ESTIMATING THE STEADY-STATE VARIANCE In the case of independent and identically distributed random variables, the well-known consistent estimate of the Proceedings 23rd European Conference on Modelling and Simulation ©ECMS Javier Otamendi, Andrzej Bargiela, José Luis Montes, Luis Miguel Doncel Pedrera (Editors) ISBN: 978-0-9553018-8-9 / ISBN: 978-0-9553018-9-6 (CD)