An Intercomparison of Sampling Methods for Uncertainty Quantification of Environmental Dynamic Models

Abstract

Uncertainty quantification (UQ) of environmental dynamic models requires an efficient way to extract the information about the relationship between input parameter and model output. A uniformly scattered sample set is generally preferred over crude Monte Carlo sampling for its ability to explore the parameter space more effectively and efficiently. This paper compares eight commonly used uniform sampling methods along with the crude Monte Carlo sampling. The efficiency is measured by six uniformity metrics, while the effectiveness is measured by the goodness-of-fit of the surrogate models, and the sensitivity analysis and optimization results. We used two test problems: the Sobol’ g-function and the SAC-SMA hydrological model. The results show that among the sampling methods evaluated, the Good Lattice Points (GLP) and Symmetric Latin hypercube (SLH) have the highest uniformity scores, and the Ranked Gram-Schmidt (RGS) de-correlation algorithm can further improve the uniformity of the lattice sample sets. On the other hand, the Quasi-Monte-Carlo (QMC) methods, such as Halton and Sobol’ sequences, are not as uniform as their theoretical potential suggests when the number of sample points is low. Further, we found no clear relationship between the sampling methods used and their effectiveness, as the latter is affected by many factors other than the sampling methods, such as the choice of the surrogate modeling methods, sensitivity analysis and optimization methods, and the intrinsic properties of the dynamic models.

Keywords: design of experiment, quasi Monte-Carlo, sampling methods, symmetric Latin hypercube, uniform design