A Quadrature Rule combining Control Variates and Adaptive Importance Sampling

Leluc, Rémi;Portier, François;Segers, Johan;Zhuman, Aigerim
(2023) Advances in Neural Information Processing Systems 35 (36th Conference on Neural Information Processing Systems - NeurIPS 2022) — ISBN: [9781713871088], p. 11842-11853, published

Files

ISBA_RP_2023-18.pdf
  • Open Access
  • Adobe PDF
  • 873.72 KB
Download

Details

Authors
  • Leluc, Rémi
    Author
  • Portier, François
    Author
  • Segers, Johanorcid-logoUCLouvain
    Author
  • Zhuman, AigerimUCLouvain
    Author
Abstract
Driven by several successful applications such as in stochastic gradient descent or in Bayesian computation, control variates have become a major tool for Monte Carlo integration. However, standard methods do not allow the distribution of the particles to evolve during the algorithm, as is the case in sequential simulation methods. Within the standard adaptive importance sampling framework, a simple weighted least squares approach is proposed to improve the procedure with control variates. The procedure takes the form of a quadrature rule with adapted quadrature weights to reflect the information brought in by the control variates. The quadrature points and weights do not depend on the integrand, a computational advantage in case of multiple integrands. Moreover, the target density needs to be known only up to a multiplicative constant. Our main result is a non-asymptotic bound on the probabilistic error of the procedure. The bound proves that for improving the estimate's accuracy, the benefits from adaptive importance sampling and control variates can be combined. The good behavior of the method is illustrated empirically on synthetic examples and real-world data for Bayesian linear regression.
Affiliations

Citations

  • APA
  • Chicago
  • FWB

Leluc, R., Portier, F., Segers, J., & Zhuman, A. (2023). A Quadrature Rule combining Control Variates and Adaptive Importance Sampling. In Ed. by S. Koyejo, S. Mohamed, A. Agarwal, D. Belgrave, K. Cho and A. Oh (ed.), Advances in Neural Information Processing Systems 35 (36th Conference on Neural Information Processing Systems - NeurIPS 2022) (p. p. 11842-11853). NeurIPS. https://doi.org/10.52202/068431-0860