  In many product development problems, the performance of the product is
governed by two types of parameters called design parameter and environmental
parameter. While the former is fully controllable, the latter varies depending
on the environment in which the product is used. The challenge of such a
problem is to find the design parameter that maximizes the probability that the
performance of the product will meet the desired requisite level given the
variation of the environmental parameter. In this paper, we formulate this
practical problem as active learning (AL) problems and propose efficient
algorithms with theoretically guaranteed performance. Our basic idea is to use
Gaussian Process (GP) model as the surrogate model of the product development
process, and then to formulate our AL problems as Bayesian Quadrature
Optimization problems for probabilistic threshold robustness (PTR) measure. We
derive credible intervals for the PTR measure and propose AL algorithms for the
optimization and level set estimation of the PTR measure. We clarify the
theoretical properties of the proposed algorithms and demonstrate their
efficiency in both synthetic and real-world product development problems.
