Evolutionary Approach to PINN Architecture Design for Approximate Solving the Laplace Equation in two Statements
with Discontinuous Initial Condition or with Measurement Data Inside a Square Domain
Abstract
Physics-informed neural networks (PINNs) are widely used today for solving differential problems and modelling physical processes described by differential equations. The paper explores the issue of PINN architecture design using evolutionary algorithms. The task of selecting suitable hyperparameter values has been set for a long time and still does not have a single approach. The article proposes a genetic algorithm for growing the size of the hidden layer of a neural network for an approximate solution of the Laplace equation in a square domain in two statements. Different variations of the evolutionary scheme are considered. The advantages and disadvantages of the parameters of these variations are discussed. The results are compared, among other things, with those obtained earlier. An original mutation procedure based on the construction of the Pareto front for various hyperparameter values in the loss function is introduced.
References
1. Raissi M., Perdikaris P., Karniadakis G.E. Physics-informed neural networks: A deep learning framework for solving forward and inverse problems involving nonlinear partial differential equations. Journal of Computational Physics. 2018;(378):686-707. https://doi.org/10.1016/j.jcp.2018.10.045
2. Tarkhov D., Vasilyev A. New neural network technique to the numerical solution of mathematical physics problems. I: Simple problems. Optical Memory and Neural Networks (Information Optics). 2005;(14):59-72.
3. Tarkhov D., Vasilyev A. New neural network technique to the numerical solution of mathematical physics problems. II: Complicated and nonstandard problems. Optical Memory and Neural Networks (Information Optics). 2005;(14):97-122.
4. Ye J., Do N.C., Zeng W., Lambert M. Physics-informed neural networks for hydraulic transient analysis in pipeline systems. Water Research. 2022;221:118828. https://doi.org/10.1016/j.watres.2022.118828
5. Bragone F., Morozovska K., Hilber P., Laneryd T., Luvisotto M. Physics-informed neural networks for modelling power transformer s dynamic thermal behaviour. Electric Power Systems Research. 2022;211:108447. https://doi.org/10.1016/j.epsr.2022.108447
6. Huang Y. Zhang Z. Zhang X. A Direct-Forcing Immersed Boundary Method for Incompressible Flows Based on Physics-Informed Neural Network. Fluids. 2022;(7):56. https://doi.org/10.3390/ fluids7020056
7. Wang Y., Han X., Chang C.-Yu., Zha D., Braga-Neto U., Hu X. Auto-PINN: Understanding and Optimizing Physics-Informed Neural Architecture. In: NeurIPS 2023 AI for Science Workshop. New Orleans, United States; 2023. 18 p. Available at: https://openreview.net/forum?id=xJFITn0hRx (accessed 24.01.2023).
8. Hornik K. Approximation capabilities of multilayer feedforward networks. Neural Networks. 1991;4(2):251-257. https://doi.org/10.1016/0893-6080(91)90009-T
9. Lazovskaya T., Tarkhov D., Dudnik A., Koksharova E., Mochalova O., Muranov D., Pozhvanyuk K., Sysoeva A. Investigation of Pareto Front of Neural Network Approximation of Solution of Laplace Equation in Two Statements: with Discontinuous Initial Conditions or with Measurement Data. In: Kryzhanovsky B., Dunin-Barkowski W., Redko V., Tiumentsev Y. (eds.) Advances in Neural Computation, Machine Learning, and Cognitive Research VI. NEUROINFORMATICS 2022. Studies in Computational Intelligence. Vol. 1064. Cham: Springer; 2023. p. 406-414. https://doi.org/10.1007/978-3-031-19032-2_42
10. Lazovskaya T., Malykhina G., Tarkhov D. Physics-Based Neural Network Methods for Solving Parameterized Singular Perturbation Problem. Computation. 2021;9(9):97. https://doi.org/10.3390/computation9090097
11. Abualigah L., Diabat A. A novel hybrid antlion optimization algorithm for multi-objective task scheduling problems in cloud computing environments. Cluster Computing. 2021;24(1):205-223. https://doi.org/10.1007/s10586-020-03075-5
12. Xiang Z., Peng W., Liu X., Yao W. Self-adaptive loss balanced Physics-informed neural networks, Neurocomputing. 2022;496:11-34. https://doi.org/10.1016/j.neucom.2022.05.015
13. Chen Z., Zhan Z., Lin Y., Gong Y., Gu T., Zhao F., Yuan H., Chen X., Li Q., Zhang J., Multiobjective Cloud Workflow Scheduling: A Multiple Populations Ant Colony System Approach. IEEE Transactions on Cybernetics. 2019;8(49);2912-2926. https://doi.org/10.1109/TCYB.2018.2832640
14. Riedmiller M., Braun H. A direct adaptive method for faster backpropagation learning: the RPROP algorithm. In: IEEE International Conference on Neural Networks. San Francisco, CA, USA: IEEE Computer Society; 1993. Vol. 1. p. 586-591. https://doi.org/10.1109/ICNN.1993.298623
15. Tarkhov D., Vasilyev A. Semi-empirical Neural Network Modeling and Digital Twins Development. Academic Press. Elsevier Publ.; 2020. 288 p. https://doi.org/10.1016/C2017-0-02027-X
16. Kuznetsov E.B., Leonov S.S., Tarkhov D.A., Vasilyev A.N. Multilayer method for solving a problem of metals rupture under creep conditions. Thermal Science. 2019;23(S2):S575-S582. https://doi.org/10.2298/TSCI19S2575K
17. Budkina E.M., Kuznetsov E.B., Tarkhov D.A., Gomzina A.A., Maltsev S.D. Comparison of solving a stiff equation on a sphere by the multi-lay-er method and method of continuing at the best parameter. Modern Information Technologies and IT-Education. 2018;14(3):533-541. (In Russ., abstract in Eng.) https://doi.org/10.25559/SITITO.14.201803.533-541
18. Kuznetsov E.B., Leonov S.S., Tarkhov D.A., Tsapko E.D., Babintseva A.A. Numerical methods for solving problems with contrast structures. Modern Information Technologies and IT-Education. 2018;14(3):542-555. (In Russ., abstract in Eng.) https://doi.org/10.25559/SITITO.14.201803.542-551
19. Kartavchenko A.E., Tarkhov D.A. Comparison of methods of construction ofapproximate analytical solutions of differential equations considering on the example of elementary functions. Modern Information Technologies and IT-Education. 2017;13(3):16-23. (In Russ., abstract in Eng.) https://doi.org/10.25559/SITITO.2017.3.440
20. Vasiliev A.N., Tarkhov D.A., Bolgov I.P., Kaverzneva T.T., Kolesova S.A., Lozovskaya T.V., Lukinsky E.V., Petrov A.A., Filkin V.M. Multilayer neural network models based on experimental data for processes of sample deformation and destruction. Modern Information Technologies and IT-Education. 2016;12(1):6-14. (In Russ., abstract in Eng.) EDN: XEQQUP
21. Tarkhov D.A., Shershneva E.A. Approximate analytical solutions of Mathieu s equations based on classical numerical methods. Modern Information Technologies and IT-Education. 2016;12(3-1):202-208. (In Russ., abstract in Eng.) EDN: XBWGWF
22. Vasiliev A.N., Lozhkin V.N., Lozhkina O.V., Tarkhov D.A., Timofeev V.D. Neural network approach in information process for predicting highway area air pollution by peat fire. Modern Information Technologies and IT-Education. 2016;12(3-2):181-187. (In Russ., abstract in Eng.) EDN: XIHNRV
23. Vasiliev A.N., Tarkhov D.A., Shemyakina T.A. Approximate analytical solutions of ordinary differential equations. Modern Information Technologies and IT-Education. 2016;12(3-2):188-195.[ 2] (In Russ., abstract in Eng.) EDN: XIHNSF
24. Kaverzneva T.T., Malykhina G.F., Tarkhov D.A. From Differential Equations to Multilayer Neural Network Models. In: Lu H., Tang H., Wang Z. (eds.) Advances in Neural Networks ISNN 2019. ISNN 2019. Lecture Notes in Computer Science. Vol. 11554. Cham: Springer; 2019. p. 19-27. https://doi.org/10.1007/978-3-030-22796-8_3
25. Zulkarnay I.U., Kaverzneva T.T., Tarkhov D.A., Tereshin V.A., Vinokhodov T.V., Kapitsin D.R. A two-layer semi-empirical model of nonlinear bending of the cantilevered beam. Journal of Physics: Conference Series Measurement Science Challenges in Natural and Social Sciences. 2017;1044:012005. https://doi.org/10.1088/1742-6596/1044/1/012005
26. Vasilyev A.N., Tarkhov D.A., Tereshin V.A., Berminova M.S., Galyautdinova A.R. Semi-empirical Neural Network Model of Real Thread Sagging. In: Kryzhanovsky B., Dunin-Barkowski W., Redko V. (eds.) Advances in Neural Computation, Machine Learning, and Cognitive Research. NEUROINFORMATICS 2017. Studies in Computational Intelligence. Vol. 736. Cham: Springer; 2018. p. 138-144. https://doi.org/10.1007/978-3-319-66604-4_21
27. Lazovskaya T., Tarkhov D., Vasilyev A. Multi-Layer Solution of Heat Equation. In: Kryzhanovsky B., Dunin-Barkowski W., Redko V. (eds.) Advances in Neural Computation, Machine Learning, and Cognitive Research. NEUROINFORMATICS 2017. Studies in Computational Intelligence. Vol. 736. Cham: Springer; 2018. p. 17-22. https://doi.org/10.1007/978-3-319-66604-4_3
28. Tarkhov D.A., Bortkovskaya M.R., Kaverzneva T.T., Kapitsin D.R., Shishkina I.A., Semenova D.A., Udalov P.P., Zulkarnay I.U. Semiempirical Model of the Real Membrane Bending. In: Kryzhanovsky B., Dunin-Barkowski W., Redko V., Tiumentsev Y. (eds.) Advances in Neural Computation, Machine Learning, and Cognitive Research II. NEUROINFORMATICS 2018. Studies in Computational Intelligence. Vol. 799. Cham: Springer; 2019. p. 221-226. https://doi.org/10.1007/978-3-030-01328-8_26
2. Tarkhov D., Vasilyev A. New neural network technique to the numerical solution of mathematical physics problems. I: Simple problems. Optical Memory and Neural Networks (Information Optics). 2005;(14):59-72.
3. Tarkhov D., Vasilyev A. New neural network technique to the numerical solution of mathematical physics problems. II: Complicated and nonstandard problems. Optical Memory and Neural Networks (Information Optics). 2005;(14):97-122.
4. Ye J., Do N.C., Zeng W., Lambert M. Physics-informed neural networks for hydraulic transient analysis in pipeline systems. Water Research. 2022;221:118828. https://doi.org/10.1016/j.watres.2022.118828
5. Bragone F., Morozovska K., Hilber P., Laneryd T., Luvisotto M. Physics-informed neural networks for modelling power transformer s dynamic thermal behaviour. Electric Power Systems Research. 2022;211:108447. https://doi.org/10.1016/j.epsr.2022.108447
6. Huang Y. Zhang Z. Zhang X. A Direct-Forcing Immersed Boundary Method for Incompressible Flows Based on Physics-Informed Neural Network. Fluids. 2022;(7):56. https://doi.org/10.3390/ fluids7020056
7. Wang Y., Han X., Chang C.-Yu., Zha D., Braga-Neto U., Hu X. Auto-PINN: Understanding and Optimizing Physics-Informed Neural Architecture. In: NeurIPS 2023 AI for Science Workshop. New Orleans, United States; 2023. 18 p. Available at: https://openreview.net/forum?id=xJFITn0hRx (accessed 24.01.2023).
8. Hornik K. Approximation capabilities of multilayer feedforward networks. Neural Networks. 1991;4(2):251-257. https://doi.org/10.1016/0893-6080(91)90009-T
9. Lazovskaya T., Tarkhov D., Dudnik A., Koksharova E., Mochalova O., Muranov D., Pozhvanyuk K., Sysoeva A. Investigation of Pareto Front of Neural Network Approximation of Solution of Laplace Equation in Two Statements: with Discontinuous Initial Conditions or with Measurement Data. In: Kryzhanovsky B., Dunin-Barkowski W., Redko V., Tiumentsev Y. (eds.) Advances in Neural Computation, Machine Learning, and Cognitive Research VI. NEUROINFORMATICS 2022. Studies in Computational Intelligence. Vol. 1064. Cham: Springer; 2023. p. 406-414. https://doi.org/10.1007/978-3-031-19032-2_42
10. Lazovskaya T., Malykhina G., Tarkhov D. Physics-Based Neural Network Methods for Solving Parameterized Singular Perturbation Problem. Computation. 2021;9(9):97. https://doi.org/10.3390/computation9090097
11. Abualigah L., Diabat A. A novel hybrid antlion optimization algorithm for multi-objective task scheduling problems in cloud computing environments. Cluster Computing. 2021;24(1):205-223. https://doi.org/10.1007/s10586-020-03075-5
12. Xiang Z., Peng W., Liu X., Yao W. Self-adaptive loss balanced Physics-informed neural networks, Neurocomputing. 2022;496:11-34. https://doi.org/10.1016/j.neucom.2022.05.015
13. Chen Z., Zhan Z., Lin Y., Gong Y., Gu T., Zhao F., Yuan H., Chen X., Li Q., Zhang J., Multiobjective Cloud Workflow Scheduling: A Multiple Populations Ant Colony System Approach. IEEE Transactions on Cybernetics. 2019;8(49);2912-2926. https://doi.org/10.1109/TCYB.2018.2832640
14. Riedmiller M., Braun H. A direct adaptive method for faster backpropagation learning: the RPROP algorithm. In: IEEE International Conference on Neural Networks. San Francisco, CA, USA: IEEE Computer Society; 1993. Vol. 1. p. 586-591. https://doi.org/10.1109/ICNN.1993.298623
15. Tarkhov D., Vasilyev A. Semi-empirical Neural Network Modeling and Digital Twins Development. Academic Press. Elsevier Publ.; 2020. 288 p. https://doi.org/10.1016/C2017-0-02027-X
16. Kuznetsov E.B., Leonov S.S., Tarkhov D.A., Vasilyev A.N. Multilayer method for solving a problem of metals rupture under creep conditions. Thermal Science. 2019;23(S2):S575-S582. https://doi.org/10.2298/TSCI19S2575K
17. Budkina E.M., Kuznetsov E.B., Tarkhov D.A., Gomzina A.A., Maltsev S.D. Comparison of solving a stiff equation on a sphere by the multi-lay-er method and method of continuing at the best parameter. Modern Information Technologies and IT-Education. 2018;14(3):533-541. (In Russ., abstract in Eng.) https://doi.org/10.25559/SITITO.14.201803.533-541
18. Kuznetsov E.B., Leonov S.S., Tarkhov D.A., Tsapko E.D., Babintseva A.A. Numerical methods for solving problems with contrast structures. Modern Information Technologies and IT-Education. 2018;14(3):542-555. (In Russ., abstract in Eng.) https://doi.org/10.25559/SITITO.14.201803.542-551
19. Kartavchenko A.E., Tarkhov D.A. Comparison of methods of construction ofapproximate analytical solutions of differential equations considering on the example of elementary functions. Modern Information Technologies and IT-Education. 2017;13(3):16-23. (In Russ., abstract in Eng.) https://doi.org/10.25559/SITITO.2017.3.440
20. Vasiliev A.N., Tarkhov D.A., Bolgov I.P., Kaverzneva T.T., Kolesova S.A., Lozovskaya T.V., Lukinsky E.V., Petrov A.A., Filkin V.M. Multilayer neural network models based on experimental data for processes of sample deformation and destruction. Modern Information Technologies and IT-Education. 2016;12(1):6-14. (In Russ., abstract in Eng.) EDN: XEQQUP
21. Tarkhov D.A., Shershneva E.A. Approximate analytical solutions of Mathieu s equations based on classical numerical methods. Modern Information Technologies and IT-Education. 2016;12(3-1):202-208. (In Russ., abstract in Eng.) EDN: XBWGWF
22. Vasiliev A.N., Lozhkin V.N., Lozhkina O.V., Tarkhov D.A., Timofeev V.D. Neural network approach in information process for predicting highway area air pollution by peat fire. Modern Information Technologies and IT-Education. 2016;12(3-2):181-187. (In Russ., abstract in Eng.) EDN: XIHNRV
23. Vasiliev A.N., Tarkhov D.A., Shemyakina T.A. Approximate analytical solutions of ordinary differential equations. Modern Information Technologies and IT-Education. 2016;12(3-2):188-195.[ 2] (In Russ., abstract in Eng.) EDN: XIHNSF
24. Kaverzneva T.T., Malykhina G.F., Tarkhov D.A. From Differential Equations to Multilayer Neural Network Models. In: Lu H., Tang H., Wang Z. (eds.) Advances in Neural Networks ISNN 2019. ISNN 2019. Lecture Notes in Computer Science. Vol. 11554. Cham: Springer; 2019. p. 19-27. https://doi.org/10.1007/978-3-030-22796-8_3
25. Zulkarnay I.U., Kaverzneva T.T., Tarkhov D.A., Tereshin V.A., Vinokhodov T.V., Kapitsin D.R. A two-layer semi-empirical model of nonlinear bending of the cantilevered beam. Journal of Physics: Conference Series Measurement Science Challenges in Natural and Social Sciences. 2017;1044:012005. https://doi.org/10.1088/1742-6596/1044/1/012005
26. Vasilyev A.N., Tarkhov D.A., Tereshin V.A., Berminova M.S., Galyautdinova A.R. Semi-empirical Neural Network Model of Real Thread Sagging. In: Kryzhanovsky B., Dunin-Barkowski W., Redko V. (eds.) Advances in Neural Computation, Machine Learning, and Cognitive Research. NEUROINFORMATICS 2017. Studies in Computational Intelligence. Vol. 736. Cham: Springer; 2018. p. 138-144. https://doi.org/10.1007/978-3-319-66604-4_21
27. Lazovskaya T., Tarkhov D., Vasilyev A. Multi-Layer Solution of Heat Equation. In: Kryzhanovsky B., Dunin-Barkowski W., Redko V. (eds.) Advances in Neural Computation, Machine Learning, and Cognitive Research. NEUROINFORMATICS 2017. Studies in Computational Intelligence. Vol. 736. Cham: Springer; 2018. p. 17-22. https://doi.org/10.1007/978-3-319-66604-4_3
28. Tarkhov D.A., Bortkovskaya M.R., Kaverzneva T.T., Kapitsin D.R., Shishkina I.A., Semenova D.A., Udalov P.P., Zulkarnay I.U. Semiempirical Model of the Real Membrane Bending. In: Kryzhanovsky B., Dunin-Barkowski W., Redko V., Tiumentsev Y. (eds.) Advances in Neural Computation, Machine Learning, and Cognitive Research II. NEUROINFORMATICS 2018. Studies in Computational Intelligence. Vol. 799. Cham: Springer; 2019. p. 221-226. https://doi.org/10.1007/978-3-030-01328-8_26
Published
2023-06-30
How to Cite
LAZOVSKAYA, Tatiana Valerievna et al.
Evolutionary Approach to PINN Architecture Design for Approximate Solving the Laplace Equation in two Statements.
Modern Information Technologies and IT-Education, [S.l.], v. 19, n. 2, p. 438-446, june 2023.
ISSN 2411-1473. Available at: <http://sitito.cs.msu.ru/index.php/SITITO/article/view/930>. Date accessed: 20 aug. 2025.
doi: https://doi.org/10.25559/SITITO.019.202302.438-446.
Section
Scientific software in education and science
This work is licensed under a Creative Commons Attribution 4.0 International License.
Publication policy of the journal is based on traditional ethical principles of the Russian scientific periodicals and is built in terms of ethical norms of editors and publishers work stated in Code of Conduct and Best Practice Guidelines for Journal Editors and Code of Conduct for Journal Publishers, developed by the Committee on Publication Ethics (COPE). In the course of publishing editorial board of the journal is led by international rules for copyright protection, statutory regulations of the Russian Federation as well as international standards of publishing.
Authors publishing articles in this journal agree to the following: They retain copyright and grant the journal right of first publication of the work, which is automatically licensed under the Creative Commons Attribution License (CC BY license). Users can use, reuse and build upon the material published in this journal provided that such uses are fully attributed.
