SOFTWARE ARCHITECTURE OF HIGH-PRODUCTION COMPLEXES FOR TEXT OF THE NEWS PROCESSING FOR SOLVING THE PROBLEMS OF DATA MINING
Abstract
The article proposes the architecture of the software for processing collections of news text messages, as well as the corresponding composition and structure of the information system. The system is a few steps of obtaining and processing information, functioning on the basis of a hybrid computing cluster. Each stage of receiving, processing and storing information in a generalized information system is represented by a microservice as a separate program unit. At the same time, the possibility of using different technology stacks for each service is emphasized so that properly selected specialized solutions increase the efficiency and quality of the result, and the shortcomings of the classical micro-service architecture are offset by the internal heterogeneity of the micro-services expressed in the form of flexible modularization. The essence of the proposed approach is the application of the principle of pipeline concurrency based on a microservice architecture with dynamic service boundaries.
References
2. Zhukov, D.O., Lesko, S.A. Stochastic self-organization of poorly structured data and memory realization in an information domain when designing news events forecasting models. The 2nd IEEE International Conference on Big Data Intelligence and Computing, Auckland, New Zealand. -- 2016.
3. Sigov, A., Zhukov, D., Novikova, O. Modelling of memory realization processes and the implementation of information self-organization in forecasting the new's events using arrays of natural language texts. 1st International Scientific Conference Convergent Cognitive Information Technologies, Moscow, Russian Federation. -- 2016.
4. A. S. Sigov, D.A. Akimov, D.O. Zhukov, E.G. Andrianova, V. E. Sachkov, V.K. Raev. Psycholinguistic analysis of text messages in Russian based on their phono semantic statistical characteristics. Informatics and applications. 2017 volume 11 issue 3, pp. 77 -86.
5. D.O. Zhukov, A.M. Zamyshlyaev, O.A. Novikova. Model of Forecasting the Social News Events on the Basis of Stochastic Dynamics Methods. ITM Web of Conferences 10, 02009 (2017) 2017 Seminar on Systems Analysis, DOI: 10.1051/itmconf/20171002009.
6. Zhukov D.O., Novikova O.A., Otradnov K.K. Methods of analysis of news events in the information space based on the use of almost – periodic functions, wavelet transforms and Hurst’s self-similarity. Proceeding The 7th International Conference on Information Communication and Management ICICM’17, August 28-30, 2017, Moscow, Russian Federation, ACM ISBN 978-1-4503-5279-6/17/08
This work is licensed under a Creative Commons Attribution 4.0 International License.
Publication policy of the journal is based on traditional ethical principles of the Russian scientific periodicals and is built in terms of ethical norms of editors and publishers work stated in Code of Conduct and Best Practice Guidelines for Journal Editors and Code of Conduct for Journal Publishers, developed by the Committee on Publication Ethics (COPE). In the course of publishing editorial board of the journal is led by international rules for copyright protection, statutory regulations of the Russian Federation as well as international standards of publishing.
Authors publishing articles in this journal agree to the following: They retain copyright and grant the journal right of first publication of the work, which is automatically licensed under the Creative Commons Attribution License (CC BY license). Users can use, reuse and build upon the material published in this journal provided that such uses are fully attributed.
