Data Visualization in Cloud Service for Scientific Computations
Abstract
The saas.jinr.ru service is an attempt to simplify the usage of the JINR Multifunctional Information and Computing Complex (MICC) of the Joint Institute for Nuclear Research (JINR). It aims at providing a simple problem-oriented web-interface to help students and beginner researchers in the physics domain to abstract away the complexity of the computing infrastructure and to focus on the actual research. In this paper we show our approach to one of the problems within the scope of the project: interactive data visualization in a web-browser. When approaching this problem, we considered two major requirements to the system: first, users may not have any programming skills, so any interaction should be performed using simple visual components; second, the system must be horizontally scalable to cope with irregular user work-sessions. The paper describes how we used Bokeh and Dask for integrating our data visualization solution within the Django framework to deal with the first requirement, and the JINR cloud for service scaling. Application of cloud technologies facilitates dynamic distribution of workload across virtual machines, thus making it possible for us to control the balance between efficient hardware utilization and end-user experience. Shared in this work resulting software architecture and applied solutions, as well as some performance considerations, can be used as an example when designing other cloud-native scientific applications.
References
2.Balashov N.A., Baranov A.V., Kutovskiy N.A., Makhalkin A.N., Mazhitova Ye.M., Pelevanyuk I.S., Semenov R.N. Present Status and Main Directions of the JINR Cloud Development. CEUR Workshop Proceedings: Proc. of 27th International Symposium NEC-2019 (Budva, Montenegro). 2019; 2507:185-189. Available at: http://ceur-ws.org/Vol-2507/185-189-paper-32.pdf (accessed 24.02.2021). (In Eng.)
3.Adam Gh., Bashashin M., Belyakov D., Kirakosyan M., Matveev M., Podgainy D., Sapozhnikova T., Streltsova O., Torosyan Sh., Vala M., Valova L., Vorontsov A., Zaikina T., Zemlyanaya E., Zuev M. IT-ecosystem of the HybriLIT heterogeneous platform for high-performance computing and training of IT-specialists. CEUR Workshop Proceedings. 2018; 2267:638-644. Available at: http://ceur-ws.org/Vol-2267/638-644-paper-122.pdf (accessed 24.02.2021). (In Eng.)
4.Balashov N., Bashashin M., Kuchumov R., Kutovskiy N., Sokolov I. JINR Cloud Service for Scientific and Engineering Computations. Sovremennye informacionnye tehnologii i IT-obrazovanie = Modern Information Technologies and IT-Education. 2018; 14(1):61-72.(In Eng.) DOI: https://doi.org/10.25559/SITITO.14.201801.061-072
5.Balashov N., Kutovskiy N., Priakhina D., Sokolov I. Evolution and Perspectives of the Service for Parallel Applications Running at JINR Multifunctional Information and Computing Complex. EPJ Web of Conferences. 2020; 226:03002. (In Eng.) DOI: https://doi.org/10.1051/epjconf/202022603002
6.Idreos S., Papaemmanouil O., Chaudhuri S. Overview of Data Exploration Techniques. Proceedings of the 2015 ACM SIGMOD International Conference on Management of Data (SIGMOD '15). Association for Computing Machinery, New York, NY, USA; 2015. p. 277-281. (In Eng.) DOI: https://doi.org/10.1145/2723372.2731084
7.Raghav R.S., Pothula S., Vengattaraman T., Ponnurangam D. A survey of data visualization tools for analyzing large volume of data in big data platform. 2016 International Conference on Communication and Electronics Systems (ICCES).Coimbatore, India; 2016. p. 1-6. (In Eng.) DOI: https://doi.org/10.1109/CESYS.2016.7889976
8.Caldarola E.G., Rinaldi A.M. Big Data Visualization Tools: A Survey. Proceedings of the 6th International Conference on Data Science, Technology and Applications (DATA 2017). SCITEPRESS – Science and Technology Publications, Lda, Setubal, PRT; 2017. p. 296-305. (In Eng.) DOI: https://doi.org/10.5220/0006484102960305
9.Qin X., Luo Y., Tang N., Li G. Making data visualization more efficient and effective: a survey. The VLDB Journal. 2020; 29(1):93-117. (In Eng.) DOI: https://doi.org/10.1007/s00778-019-00588-3
10.Shukrinov Yu.M. et al. Modeling of LC-shunted intrinsic Josephson junctions in high-Tc superconductors. Superconductor Science and Technology. 2016; 30(2):024006. (In Eng.) DOI: https://doi.org/10.1088/1361-6668/30/2/024006
11.Bokhari M.U., Shallal Q.M., Tamandani Y.K. Cloud computing service models: A comparative study. 2016 3rd International Conference on Computing for Sustainable Global Development (INDIACom). IEEE, New Delhi, India; 2016. p. 890-895. (In Eng.)
12.Cardellini V., Colajanni M., Yu P.S. Dynamic load balancing on Web-server systems. IEEE Internet Computing. 1999; 3(3):28-39. (In Eng.) DOI: https://doi.org/10.1109/4236.769420
13.Rocklin M. Dask: Parallel Computation with Blocked algorithms and Task Scheduling. In: K. Huff, J. Bergstra (Eds.) Proceedings of the 14th Python in Science Conference (SciPy 2015). Austin, Texas; 2015. p. 126-132. (In Eng.) DOI: https://doi.org/10.25080/Majora-7b98e3ed-013
14.Bird I. Computing for the Large Hadron Collider. Annual Review of Nuclear and Particle Science. 2011; 61:99-118. (In Eng.) DOI: https://doi.org/10.1146/annurev-nucl-102010-130059
15.Baranov A.V., Balashov N.A., Kutovskiy N.A., Semenov R.N. JINR cloud infrastructure evolution. Physics of Particles and Nuclei Letters. 2016; 13(5):672-675. (In Eng.) DOI: https://doi.org/10.1134/S1547477116050071
16.Greenfeld D.R., Greenfeld A.R. Two Scoops of Django: Best Practices for Django 1.8. 3rd ed. Two Scoops Press. Publ.; 2015. (In Eng.)
17.Thain D., Tannenbaum T., Livny M. Distributed Computing in Practice: The Condor Experience. Concurrency and Computation: Practice and Experience. 2005; 17(2-4):323-356. (In Eng.) DOI: https://doi.org/10.1002/cpe.938
18.Adamson P. et al. First measurement of electron neutrino appearance in NovA. Physical Review Letters. 2016; 116(15):151806. (In Eng.) DOI: https://doi.org/10.1103/PhysRevLett.116.151806
19.Moreno-Vozmediano R., Montero R.S., Llorente I.M. IaaS Cloud Architecture: From Virtualized Datacenters to Federated Cloud Infrastructures. IEEE Computer. 2012; 45(12):65-72. (In Eng.) DOI: https://doi.org/10.1109/MC.2012.76
20.Balashov N.A. et al. JINR Member States cloud infrastructure. CEUR Workshop Proceedings. 2017; 2023:202-206. Available at: http://ceur-ws.org/Vol-2023/122-128-paper-19.pdf (accessed 24.02.2021). (In Eng.)
21.Balashov N.A. et al. Service for parallel applications based on JINR cloud and HybriLIT resources. EPJ Web of Conferences. 2019; 214:07012. (In Eng.) DOI: https://doi.org/10.1051/epjconf/201921407012
22.Balashov N. et al. Creating a Unified Educational Environment for Training IT Specialists of Organizations of the JINR Member States in the Field of Cloud Technologies. In: V. Sukhomlin, E. Zubareva (Eds.) Modern Information Technology and IT Education. SITITO 2018. Communications in Computer and Information Science, vol. 1201. Springer, Cham; 2020. p. 149-162. (In Eng.) DOI: https://doi.org/10.1007/978-3-030-46895-8_12
23.Goncharov P., Ososkov G., Nechaevskiy A., Uzhinskiy A. Architecture and basic principles of the multifunctional platform for plant disease detection. CEUR Workshop Proceedings. 2018; 2267:200-206. Available at: http://ceur-ws.org/Vol-2267/200-206-paper-37.pdf (accessed 24.02.2021). (In Eng.)
24.Weil S.A., Brandt S.A., Miller E.L., Long D.D.E., MaltzahnC. Ceph: a scalable, high-performance distributed file system. Proceedings of the 7th symposium on Operating systems design and implementation (OSDI'06). USENIX Association, Berkeley, CA, USA; 2006. p. 307-320. Available at: https://www.ssrc.ucsc.edu/papers/weil-osdi06.pdf (accessed 24.02.2021). (In Eng.)
25.Massie M., Chun B., Culler D. The Ganglia Distributed Monitoring System: Design, Implementation, and Experience. Parallel Computing. 2004; 30:817-840. (In Eng.) DOI: https://doi.org/10.1016/j.parco.2004.04.001
This work is licensed under a Creative Commons Attribution 4.0 International License.
Publication policy of the journal is based on traditional ethical principles of the Russian scientific periodicals and is built in terms of ethical norms of editors and publishers work stated in Code of Conduct and Best Practice Guidelines for Journal Editors and Code of Conduct for Journal Publishers, developed by the Committee on Publication Ethics (COPE). In the course of publishing editorial board of the journal is led by international rules for copyright protection, statutory regulations of the Russian Federation as well as international standards of publishing.
Authors publishing articles in this journal agree to the following: They retain copyright and grant the journal right of first publication of the work, which is automatically licensed under the Creative Commons Attribution License (CC BY license). Users can use, reuse and build upon the material published in this journal provided that such uses are fully attributed.
