Sampling of integrand for integration using shallow neural network

Cover Page

Cite item

Full Text

Abstract

Inthispaper,westudytheeffectofusingtheMetropolis-Hastingsalgorithmforsamplingtheintegrand on the accuracy of calculating the value of the integral with the use of shallow neural network. In addition, a hybrid method for sampling the integrand is proposed, in which part of the training sample is generated by applying the Metropolis-Hastings algorithm, and the other part includes points of a uniform grid. Numerical experiments show that when integrating in high-dimensional domains, sampling of integrands both by the Metropolis-Hastings algorithm and by a hybrid method is more efficient with respect to the use of a uniform grid.

Full Text

1. Introduction In the recent study [1], an algorithm for numerical integration was proposed based on the use of a neural network with one hidden layer. In this approach, the neural network approximates the integrand function within a bounded region that includes the integration domain. A training the neural network may certainly require a significant amount of time and computational resources. However, when the training is completed, the neural network architecture allows for the analytical integration of the approximated integrand. Furthermore, the integral of the neural network’s function can be computed in any other subregion without the need for retraining. Thus, the neural network integration approach is efficient for tasks where it is necessary to repeatedly calculate the integral of the same function in different regions. The neural network training is the main challenge of an integrand approximation. During supervised learning training data plays a significant role, and consequently an approach to their sampling. In the paper [1], a uniform grid-based discretization of the domain was used as the function sampling method. However, this approach is inefficient for integrands with significant variations in certain subregions. In this article, the impact of using the Metropolis-Hastings algorithm for shape-based sampling of an integrand on the integration accuracy is considered. A hybrid approach for forming the training dataset is proposed, in which a portion of the training dataset (with a relative volume fraction denoted as
×

About the authors

Alexander S. Ayriyan

Joint Institute for Nuclear Research; Alikhanyan National Science Laboratory; Dubna State University

Email: ayriyan@jinr.ru
ORCID iD: 0000-0002-5464-4392

PhD in Physics and Mathematics, Head of sector of the Division of Computational Physics of JINR, Assistant professor of Department of Distributed Information Computing Systems of Dubna State University; Senior Researcher of AANL

6 Joliot-Curie St, Dubna, 141980, Russian Federation; 2 Alikhanyan Brothers St, Yerevan, 0036, Republic of Armenia; 19 Universitetskaya St, Dubna, 141980, Russian Federation

Hovik A. Grigorian

Joint Institute for Nuclear Research; Alikhanyan National Science Laboratory; Dubna State University; Yerevan State University

Email: hovik.grigorian@gmail.com
ORCID iD: 0000-0002-0003-0512

Candidate of Physical and Mathematical Sciences, Senior Researcher of JINR; Senior Researcher of AANL (YerPhI); Assistant professor of Dubna State University; assistant professor of Yerevan State University

6 Joliot-Curie St, Dubna, 141980, Russian Federation; 2 Alikhanyan Brothers St, Yerevan, 0036, Republic of Armenia; 19 Universitetskaya St, Dubna, 141980, Russian Federation; 1 Alex Manoogian St, Yerevan, 0025, Republic of Armenia

Vladimir V. Papoyan

Joint Institute for Nuclear Research; Alikhanyan National Science Laboratory; Dubna State University

Author for correspondence.
Email: vlpapoyan@jinr.ru
ORCID iD: 0000-0003-0025-5444

Junior researcher of JINR, Junior researcher of AANL (YerPhI), PhD student of Dubna State University

6 Joliot-Curie St, Dubna, 141980, Russian Federation; 2 Alikhanyan Brothers St, Yerevan, 0036, Republic of Armenia; 19 Universitetskaya St, Dubna, 141980, Russian Federation

References

  1. Lloyd, S., Irani, R. A. & Ahmadi, M. Using neural networks for fast numerical integration and optimization. IEEE Access 8, 84519-84531. doi: 10.1109/access.2020.2991966 (2020).
  2. Cybenko, G. Approximation by superpositions of a sigmoidal function. Mathematics of Control Signals and Systems 2, 303-314. doi: 10.1007/bf02551274 (Dec. 1989).
  3. Hastings, W. K. Monte Carlo sampling methods using Markov chains and their applications. Biometrika 57, 97-109. doi: 10.1093/biomet/57.1.97 (Apr. 1970).
  4. Chib, S. & Greenberg, E. Understanding the Metropolis-Hastings algorithm. The American Statistician 49, 327. doi: 10.2307/2684568 (Nov. 1995).
  5. Ecosystem for tasks of machine learning, deep learning and data analysis http://hlit.jinr.ru/en/access-to-resources_eng/ecosystem-for-ml_dl_bigdataanalysistasks_eng/. Accessed: 2023-10-10.
  6. Chollet, F. et al. Keras https://keras.io.
  7. Johansson, F. et al. mpmath: a Python library for arbitrary-precision floating-point arithmetic (version 0.18) http://mpmath.org.
  8. Marquardt, D. W. An algorithm for least-squares estimation of nonlinear parameters. Journal of the Society for Industrial and Applied Mathematics 11, 431-441. doi: 10.1137/0111030 (June 1963).
  9. Marco, F. D. Tensorflow Levenberg-Marquardt https://github.com/fabiodimarco/tflevenberg-marquardt.
  10. Kişi, Ö. & Uncuoǧlu, E. Comparison of three back-propagation training algorithms for two case studies. Indian Journal of Engineering and Materials Sciences 12, 434-442 (Oct. 2005).
  11. Jiawei Han, M. K. & Pei, J. Data mining: concepts and techniques Third Edition. 703 pp. doi: 10.1016/c2009-0-61819-5 (Elsevier Inc., 225 Wyman Street, Waltham, MA 02451, USA, 2012).
  12. Genz, A. A package for testing multiple integration subroutines in Numerical Integration 337-340 (Springer Netherlands, 1987). doi: 10.1007/978-94-009-3889-2_33.

Copyright (c) 2024 Ayriyan A.S., Grigorian H.A., Papoyan V.V.

Creative Commons License
This work is licensed under a Creative Commons Attribution-NonCommercial 4.0 International License.

This website uses cookies

You consent to our cookies if you continue to use our website.

About Cookies