Discrete and Continuous Models and Applied Computational ScienceDiscrete and Continuous Models and Applied Computational ScienceDiscrete and Continuous Models and Applied Computational Science2658-46702658-7149Peoples' Friendship University of Russia named after Patrice Lumumba (RUDN University)2291310.22363/2658-4670-2019-27-4-305-315Computer ScienceИнформатика и вычислительная техникаResearch ArticleReview and comparative analysis of machine learning libraries for machine learningОбзор и сравнительный анализ библиотек машинного обучения для построения нейронных сетейGevorkyanMigran N.ГеворкянМ. Н.

Candidate of Physical and Mathematical Sciences, assistant professor of Department of Applied Probability and Informatics

Кафедра прикладной информатики и теории вероятностей

gevorkyan-mn@rudn.ruDemidovaAnastasia V.ДемидоваА. В.

Candidate of Physical and Mathematical Sciences, assistant professor of Department of Applied Probability and Informatics

Кафедра прикладной информатики и теории вероятностей

demidova-av@rudn.ruDemidovaTatiana S.ДемидоваТ. С.

student of Department of Applied Probability and Informatics

Кафедра прикладной информатики и теории вероятностей

1032152607@pfur.ruSobolevAnton A.СоболевА. А.

student of Department of Applied Probability and Informatics

Кафедра прикладной информатики и теории вероятностей

1032152618@pfur.ruPeoples’ Friendship University of RussiaРоссийский университет дружбы народов15122019274VOL 27, NO4 (2019)ТОМ 27, №4 (2019)30531519022020Copyright ©; 2019, Gevorkyan M.N., Demidova A.V., Demidova T.S., Sobolev A.A.Copyright ©; 2019, Геворкян М.Н., Демидова А.В., Демидова Т.С., Соболев А.А.2019Gevorkyan M.N., Demidova A.V., Demidova T.S., Sobolev A.A.Геворкян М.Н., Демидова А.В., Демидова Т.С., Соболев А.А.http://creativecommons.org/licenses/by/4.0https://journals.rudn.ru/miph/article/view/22913

The article is an overview. We carry out the comparison of actual machine learning libraries that can be used the neural networks development. The first part of the article gives a brief description of TensorFlow, PyTorch, Theano, Keras, SciKit Learn libraries, SciPy library stack. An overview of the scope of these libraries and the main technical characteristics, such as performance, supported programming languages, the current state of development is given. In the second part of the article, a comparison of five libraries is carried out on the example of a multilayer perceptron, which is applied to the problem of handwritten digits recognizing. This problem is well known and well suited for testing different types of neural networks. The study time is compared depending on the number of epochs and the accuracy of the classifier. The results of the comparison are presented in the form of graphs of training time and accuracy depending on the number of epochs and in tabular form.

Статья носит обзорный характер. В ней проведено сравнение актуальных библиотек машинного обучения, которые могут быть использованы для построения нейронных сетей. В первой части статьи даётся краткое описание библиотек TensorFlow, PyTorch, Theano, Keras, SciKit Learn, стека библиотек SciPy (NumPy, SciPy, Pandas, Matplotlib, Jupyter). Делается обзор области применения перечисленных библиотек и основных технических характеристик, таких как быстродействие, поддерживаемые языки программирования, текущее состояние разработки. Среди рассматриваемых библиотек только PyTorch и TensorFlow непосредственно конкурируют друг с другом. Остальные библиотеки взаимодополняют друг друга и часто используются совместно при построении различных моделей машинного обучения. Во второй части статьи проводится сравнение пяти библиотек на примере многослойного перцептрона, который применяется к задаче распознания рукописных цифр. Данная задача хорошо разработана и является модельной для тестирования различных реализаций нейронных сетей. Сравнивается время обучения в зависимости от количества эпох и точности работы классификатора. Результаты сравнения представлены в виде графиков времени обучения и точности в зависимости от количества эпох и в табличном виде.

machine learningneural networksMNISTTensorFlowPyTorchMNISTTensorFlowPyTorchмашинное обучениенейронные сетиThe publication was prepared with the support of the “RUDN University Program 5-100”.The publication was prepared with the support of the “RUDN University Program 5-100”.G. Van Rossum and F. L. Drake Jr, Python tutorial. Centrum voor Wiskunde en Informatica Amsterdam, The Netherlands, 1995.T. Kluyver et al., “Jupyter Notebooks - a publishing format for repro- ducible computational workflows,” in Positioning and Power in Academic Publishing: Players, Agents and Agendas, F. Loizides and B. Schmidt, Eds., IOS Press, 2016, pp. 87-90. DOI: 10.3233/978-1-61499-649-1- 87.J. D. Hunter, “Matplotlib: A 2D graphics environment,” Computing in Science & Engineering, vol. 9, no. 3, pp. 90-95, 2007. DOI: 10.1109/ MCSE.2007.55.F. Pérez and B. E. Granger, “IPython: a system for interactive scientific computing,” Computing in Science and Engineering, vol. 9, no. 3, pp. 21- 29, May 2007. DOI: 10.1109/MCSE.2007.53.S. Behnel, R. Bradshaw, C. Citro, L. Dalcin, D. S. Seljebotn, and K. Smith, “Cython: the best of both worlds,” Computing in Science & Engineering, vol. 13, no. 2, pp. 31-39, Mar. 2011. DOI: 10.1109/MCSE. 2010.118.S. van der Walt, S. C. Colbert, and G. Varoquaux, “The NumPy array: a structure for efficient numerical computation,” Computing in Science Engineering, vol. 13, no. 2, pp. 22-30, Mar. 2011. DOI: 10.1109/MCSE. 2011.37.E. Jones, T. Oliphant, P. Peterson, et al. (2001-). SciPy: open source scientific tools for Python, [Online]. Available: http:// www. scipy. org/.W. McKinney, “Data structures for statistical computing in Python,” in Proceedings of the 9th Python in Science Conference, S. van der Walt and J. Millman, Eds., 2010, pp. 51-56.[9] Martıń Abadi et al. (2015). TensorFlow: large-scale machine learning on heterogeneous systems. Software available from tensorflow.org, [Online]. Available: http://tensorflow.org/.(2019). TensorFlow official repository, [Online]. Available: https:// github.com/tensorflow/tensorflow.A. Paszke, S. Gross, S. Chintala, G. Chanan, E. Yang, Z. DeVito, Z. Lin, Desmaison, L. Antiga, and A. Lerer, “Automatic differentiation in PyTorch,” in 31st Conference on Neural Information Processing Systems (NIPS 2017), Long Beach, CA, USA, 2017.(2019). Torch official repository, [Online]. Available: https://github. com/torch/torch7.Theano Development Team, “Theano: a Python framework for fast computation of mathematical expressions,” May 2016. eprint: arXiv: abs/1605.02688.F. Chollet. (2019). Keras, [Online]. Available: https://keras.io/.(2019). CNTC official repository, [Online]. Available: https://github. com/Microsoft/cntk.F. Pedregosa et al., “Scikit-learn: machine learning in Python,” Journal of Machine Learning Research, vol. 12, pp. 2825-2830, 2011.(2019). MNIST handwritten digit database, Yann LeCun, Corinna Cortes and Chris Burges, [Online]. Available: http:// yann. lecun. com/exdb/mnist/.F. Rosenblatt, “The perceptron: a probabilistic model for information storage and organization in the brain,” Psychological review, vol. 65, no. 6, 1958. DOI: 10.1037/h0042519.X. Glorot, A. Bordes, and Y. Bengio, “Deep Sparse Rectifier Neural Networks,” in Proceedings of the Fourteenth International Conference on Artificial Intelligence and Statistics, G. Gordon, D. Dunson, and M. Dudík, Eds., ser. Proceedings of Machine Learning Research, vol. 15, Fort Lauderdale, FL, USA: PMLR, Nov. 2011, pp. 315-323.T. Zhang, “Solving large scale linear prediction problems using sto- chastic gradient descent algorithms,” in Proceedings of the Twenty-first International Conference on Machine Learning, ser. ICML ’04, Banff, Alberta, Canada: ACM, 2004. DOI: 10.1145/1015330.1015332.