Оптимизация аналитических запросов в гетерогенных системах
Аннотация
Согласно оценке International Data Corporation (IDC) в 2020 году человечество произвело более 64 зеттабайт данных. В ближайшие 5 лет ожидается рост этого показателя на 23% в год, что превышает скорость роста производительности аппаратуры. Агрегация и анализ больших данных потребовали развития таких технологий как Hadoop и Spark. С увеличением количества данных и сложности запросов их производительности оказывается недостаточно и требуются новые пути повышения эффективности анализа. Один из таких методов – использование специализированных аппаратных ускорителей, таких как графический процессор, для выполнения запросов и расширение иерархии используемой памяти. В статье рассматривается проблема оптимизации аналитических запросов к СУБД в основной памяти с помощью использования аппаратных ускорителей и расширения иерархии памяти. Дан обзор классических подходов к оптимизации запросов и текущего состояния области исследования гетерогенного исполнения. Разобраны достоинства и недостатки существующих решений и сформулированы нерешенные проблемы. Предложен вариант эталонной архитектуры.
Литература
1. Wang C., Chen M.-S. On the complexity of distributed query optimization. IEEE Transactions on Knowledge and Data Engineering. 1996; 8(4):650-662. (In Eng.) doi: https://doi.org/10.1109/69.536256
2. Appuswamy R., Karpathiotakis M., Porobic D., Ailamaki A. The Case For Heterogeneous HTAP. Proceedings of the 8th Biennial Conference on Innovative Data Systems Research (CIDR'17). Chaminade, California, USA; 2017. Available at: https://www.cidrdb.org/cidr2017/papers/p21-appuswamy-cidr17.pdf (accessed 13.10.2021). (In Eng.)
3. Ding B., Das S., Wu W., Chaudhuri S., Narasayya V. Plan Stitch: Harnessing the Best of Many Plans. Proceedings of the VLDB Endowment. 2018; 11(10):1123-1136. (In Eng.) doi: https://doi.org/10.14778/3231751.3231761
4. Giannikis G., Makreshanski D., Alonso G., Kossmann D. Shared Workload Optimization. Proceedings of the VLDB Endowment. 2014; 7(6):429-440. (In Eng.) doi: https://doi.org/10.14778/2732279.2732280
5. Mishra A., et al. Accelerating Analytics with Dynamic In-Memory Expressions. Proceedings of the VLDB Endowment. 2016; 9(13):1437-1448. (In Eng.) doi: https://doi.org/10.14778/3007263.3007280
6. Barthels C., Müller I., Schneider T., Alonso G., Hoefler T. Distributed Join Algorithms on Thousands of Cores. Proceedings of the VLDB Endowment. 2017; 10(5):517-528. (In Eng.) doi: https://doi.org/10.14778/3055540.3055545
7. Graefe G. Encapsulation of Parallelism in the Volcano Query Processing System. Proceedings of the 1990 ACM SIGMOD International Conference on Management of Data (Atlantic City, New Jersey, USA) (SIGMOD'90). Association for Computing Machinery, New York, NY, USA; 199. p. 102-111. (In Eng.) doi: https://doi.org/10.1145/93597.98720
8. Shaikhha A., Klonatos Ya., Parreaux L., Brown L., Dashti M., Koch C. How to Architect a Query Compiler. Proceedings of the 2016 International Conference on Management of Data (San Francisco, California, USA) (SIGMOD'16). Association for Computing Machinery, New York, NY, USA; 2016. p. 1907-1922. (In Eng.) doi: https://doi.org/10.1145/2882903.2915244
9. DeWitt D.J., Katz R.H., Olken F., Shapiro L.D., Stonebraker M.R., Wood D.A. Implementation Techniques for Main Memory Database Systems. ACM SIGMOD Record. 1984; 14(2):1-8. (In Eng.) doi: https://doi.org/10.1145/971697.602261
10. Stonebraker M., Cetintemel U. "One Size Fits All": An Idea Whose Time Has Come and Gone. Proceedings of the 21st International Conference on Data Engineering (ICDE'05). IEEE Computer Society, USA; 2005. p. 2-11. (In Eng.) doi: https://doi.org/10.1109/ICDE.2005.1
11. Ailamaki A., DeWitt D.J., Hill M.D., Wood D.A. DBMSs on a Modern Processor: Where Does Time Go? Proceedings of the 25th International Conference on Very Large Data Bases (VLDB'99). Morgan Kaufmann Publishers Inc., San Francisco, CA, USA; 1999. p. 266-277. Available at: https://www.vldb.org/conf/1999/P28.pdf (accessed 13.10.2021). (In Eng.)
12. Kester M.S., Athanassoulis M., Idreos S. Access Path Selection in Main-Memory Optimized Data Systems: Should I Scan or Should I Probe? Proceedings of the 2017 ACM International Conference on Management of Data (Chicago, Illinois, USA) (SIGMOD'17). Association for Computing Machinery, New York, NY, USA; 2017. p. 715-730. (In Eng.) doi: https://doi.org/10.1145/3035918.3064049
13. Neumann T. Efficiently Compiling Efficient Query Plans for Modern Hardware. Proceedings of the VLDB Endowment. 2011; 4(9):539-550. (In Eng.) doi: https://doi.org/10.14778/2002938.2002940
14. Kersten T., Leis V., Kemper A., Neumann T., Pavlo A., Boncz P. Everything You Always Wanted to Know about Compiled and Vectorized Queries but Were Afraid to Ask. Proceedings of the VLDB Endowment. 2018; 11(13):2209-2222. (In Eng.) doi: https://doi.org/10.14778/3275366.3284966
15. Lottarini A., Ramirez A., Coburn J., Kim M.A., Ranganathan P., Stodolsky D., Wachsler M. Vbench: Benchmarking Video Transcoding in the Cloud. ACM SIGPLAN Notices. 2018; 53(2):797-809. (In Eng.) doi: https://doi.org/10.1145/3296957.3173207
16. Kara K., Giceva J., Alonso G. FPGA-Based Data Partitioning. Proceedings of the 2017 ACM International Conference on Management of Data (Chicago, Illinois, USA) (SIGMOD'17). Association for Computing Machinery, New York, NY, USA; 2017. p. 433-445. (In Eng.) doi: https://doi.org/10.1145/3035918.3035946
17. Karnagel T., Habich D., Schlegel B., Lehner W. Heterogeneity-Aware Operator Placement in Column-Store DBMS. Datenbank-Spektrum. 2014; 14(3):211-221. (In Eng.) doi: https://doi.org/10.1007/s13222-014-0167-9
18. Müller M., Leich T., Pionteck T., Saake G., Teubner J., Spinczyk O. He..ro DB: A Concept for Parallel Data Processing on Heterogeneous Hardware. In: Brinkmann A., Karl W., Lankes S., Tomforde S., Pionteck T., Trinitis C. (eds.) Architecture of Computing Systems – ARCS 2020. ARCS 2020. Lecture Notes in Computer Science. Vol. 12155. Springer, Cham; 2020. p. 82096. (In Eng.) doi: https://doi.org/10.1007/978-3-030-52794-5_7
19. Zhang Ya., Zhang Yu, Lu J., Wang Sh., Liu Z., Han R. One size does not fit all: accelerating OLAP workloads with GPUs. Distributed and Parallel Databases. 2020; 38(4):995-1037. (In Eng.) doi: https://doi.org/10.1007/s10619-020-07304-z
20. Barroso L., Marty M., Patterson D., Ranganathan P. Attack of the Killer Microseconds. Communications of the ACM. 2017; 60(4):48-54. (In Eng.) doi: https://doi.org/10.1145/3015146
21. Chernishev G.A., Galaktionov V.A., Grigorev V.D., Klyuchikov E.S., Smirnov K.K. PosDB: An Architecture Overview. Programming and Computer Software. 2018; 44(1):62-74. (In Eng.) doi: https://doi.org/10.1134/S0361768818010024
22. Funke H., Breß S., Noll S., Markl V., Teubner J. Pipelined Query Processing in Coprocessor Environments. Proceedings of the 2018 International Conference on Management of Data (Houston, TX, USA) (SIGMOD'18). Association for Computing Machinery, New York, NY, USA; 2018. p. 1603-1618. (In Eng.) doi: https://doi.org/10.1145/3183713.3183734
23. Răducanu B., Boncz P., Zukowski M. Micro Adaptivity in Vectorwise. Proceedings of the 2013 ACM SIGMOD International Conference on Management of Data (New York, New York, USA) (SIGMOD'13). Association for Computing Machinery, New York, NY, USA; 2013. p. 1231-1242. (In Eng.) doi: https://doi.org/10.1145/2463676.2465292
24. Leis V., Boncz P., Kemper A., Neumann T. Morsel-Driven Parallelism: A NUMA-Aware Query Evaluation Framework for the Many-Core Age. Proceedings of the 2014 ACM SIGMOD International Conference on Management of Data (Snowbird, Utah, USA) (SIGMOD'14). Association for Computing Machinery, New York, NY, USA; 2014. p. 743-754. (In Eng.) doi: https://doi.org/10.1145/2588555.2610507
25. Diaconu C., Freedman C., Ismert E., Larson P.-A., Mittal P., Stonecipher R., Verma N., Zwilling M. Hekaton: SQL Server’s Memory-Optimized OLTP Engine. Proceedings of the 2013 ACM SIGMOD International Conference on Management of Data (New York, New York, USA) (SIGMOD'13). Association for Computing Machinery, New York, NY, USA; 2013. p. 1243-1254. (In Eng.) doi: https://doi.org/10.1145/2463676.2463710
26. Chen J., Jindel S., Walzer R., Sen R., Jimsheleishvilli N., Andrews M. The MemSQL Query Optimizer: A Modern Optimizer for Real-Time Analytics in a Distributed Database. Proceedings of the VLDB Endowment. 2016; 9(13):1401-1412. (In Eng.) doi: https://doi.org/10.14778/3007263.3007277
27. Boncz P., Zukowski M., Nes N. MonetDB/X100: Hyper-Pipelining Query Execution. Proceedings of the Second Biennial Conference on Innovative Data Systems Research (CIDR'05). Asilomar, CA, USA; 2005. Available at: http://cidrdb.org/cidr2005/papers/P19.pdf (accessed 13.10.2021). (In Eng.)
28. Psaroudakis I., Scheuer T., May N., Sellami A., Ailamaki A. Scaling up Concurrent Main-Memory Column-Store Scans: Towards Adaptive NUMA-Aware Data and Task Placement. Proceedings of the VLDB Endowment. 2015; 8(12):1442-1453. (In Eng.) doi: https://doi.org/10.14778/2824032.2824043
29. Graefe G. The Cascades Framework for Query Optimization. IEEE Data Engineering Bulletin. 1995; 18(3):19-29. Available at: https://www.cse.iitb.ac.in/infolab/Data/Courses/CS632/Papers/Cascades-graefe.pdf (accessed 13.10.2021). (In Eng.)
30. Leis V., Gubichev A., Mirchev A., Boncz P., Kemper A., Neumann T. How Good Are Query Optimizers, Really? Proceedings of the VLDB Endowment. 2015; 9(3):204-215. (In Eng.) doi: https://doi.org/10.14778/2850583.2850594
31. Cormode G., Garofalakis M., Haas P.J., Jermaine C. Synopses for Massive Data: Samples, Histograms, Wavelets, Sketches. Foundations and Trends in Databases. 2012; 4(1-3):1-294. (In Eng.) doi: https://doi.org/10.1561/1900000004
32. Sun J., Li G. An End-to-End Learning-Based Cost Estimator. Proceedings of the VLDB Endowment. 2 019; 13(3):307-319. (In Eng.) doi: https://doi.org/10.14778/3368289.3368296
33. Marcus R., Negi P., Mao H., Zhang C., Alizadeh M., Kraska T., Papaemmanouil O., Tatbul N. Neo: A Learned Query Optimizer. Proceedings of the VLDB Endowment. 2019; 12(11):1705-1718. (In Eng.) doi: https://doi.org/10.14778/3342263.3342644
34. Negi P., Interlandi M., Marcus R., Alizadeh M., Kraska T., Friedman M., Jindal A. Steering Query Optimizers: A Practical Take on Big Data Workloads. Proceedings of the 2021 International Conference on Management of Data (Virtual Event, China) (SIGMOD/PODS'21). Association for Computing Machinery, New York, NY, USA; 2021. p. 2557-2569. (In Eng.) doi: https://doi.org/10.1145/3448016.3457568
35. Horng J.-T., Kao C.-Y., Liu B.-J. A genetic algorithm for database query optimization. Proceedings of the First IEEE Conference on Evolutionary Computation. IEEE World Congress on Computational Intelligence. Vol. 1. IEEE Computer Society; 1994. p. 350-355. (In Eng.) doi: https://doi.org/10.1109/ICEC.1994.349926
36. Lutz C., Breß S., Zeuch S., Rabl T., Markl V. Pump Up the Volume: Processing Large Data on GPUs with Fast Interconnects. Proceedings of the 2020 ACM SIGMOD International Conference on Management of Data (Portland, OR, USA) (SIGMOD'20). Association for Computing Machinery, New York, NY, USA; 2020. p. 1633-1649. (In Eng.) doi: https://doi.org/10.1145/3318464.3389705
37. Shanbhag A., Madden S., Yu X. A Study of the Fundamental Performance Characteristics of GPUs and CPUs for Database Analytics. Proceedings of the 2020 ACM SIGMOD International Conference on Management of Data (Portland, OR, USA) (SIGMOD'20). Association for Computing Machinery, New York, NY, USA; 2020. p. 1617-1632. (In Eng.) doi: https://doi.org/10.1145/3318464.3380595
38. He B., Lu M., Yang K., Fang R., Govindaraju N.K., Luo Q., Sander P.V. Relational Query Coprocessing on Graphics Processors. ACM Transactions on Database Systems. 2009; 34(4):21. (In Eng.) doi: https://doi.org/10.1145/1620585.1620588
39. Heimel M., Saecker M., Pirk H., Manegold S., Markl V. Hardware-Oblivious Parallelism for in-Memory Column-Stores. Proceedings of the VLDB Endowment. 2013; 6(9):709-720. (In Eng.) doi: https://doi.org/10.14778/2536360.2536370
40. Stone J.E., Gohara D., Shi G. OpenCL: A Parallel Programming Standard for Heterogeneous Computing Systems. Computing in Science Engineering. 2010; 12(3):66-73. (In Eng.) doi: https://doi.org/10.1109/MCSE.2010.69
41. Breß S., Köcher B., Funke H., Zeuch S., Rabl T., Markl V. Generating Custom Code for Efficient Query Execution on Heterogeneous Processors. The VLDB Journal. 2018; 27(6):797-822. (In Eng.) doi: https://doi.org/10.1007/s00778-018-0512-y
42. Kohn A., Leis V., Neumann T. Making Compiling Query Engines Practical. IEEE Transactions on Knowledge and Data Engineering. 2021; 33(2):597-612. (In Eng.) doi: https://doi.org/10.1109/TKDE.2019.2905235
43. Pirk H., Moll O., Zaharia M., Madden S. Voodoo ‒ a Vector Algebra for Portable Database Performance on Modern Hardware. Proceedings of the VLDB Endowment. 2016; 9(14):1707-1718. (In Eng.) doi: https://doi.org/10.14778/3007328.3007336
44. Shaikhha A., Klonatos Ya., Koch C. Building Efficient Query Engines in a High-Level Language. ACM Transactions on Database Systems. 2018; 43(1):4. (In Eng.) doi: https://doi.org/10.1145/3183653
45. Chrysogelos P., Karpathiotakis M., Appuswamy R., Ailamaki A. HetExchange: Encapsulating Heterogeneous CPU-GPU Parallelism in JIT Compiled Engines. Proceedings of the VLDB Endowment. 2019; 12(5):544-556. (In Eng.) doi: https://doi.org/10.14778/3303753.3303760
46. Gubner T., Boncz P. Charting the Design Space of Query Execution Using VOILA. Proceedings of the VLDB Endowment. 2021; 14(6):1067-1079. (In Eng.) doi: https://doi.org/10.14778/3447689.3447709
47. Palkar S., Thomas J.J., Shanbhag A., Narayanan D., Pirk H., Schwarzkopf M., Amarasinghe S., Zaharia M. Weld: A Common Runtime for High Performance Data Analytics. Proceedings of the 8th Biennial Conference on Innovative Data Systems Research (CIDR'17). Chaminade, California, USA; 2017. Available at: https://www.cidrdb.org/cidr2017/papers/p127-palkar-cidr17.pdf (accessed 13.10.2021). (In Eng.)
48. Zhang S., He J., He B., Lu M. OmniDB: Towards Portable and Efficient Query Processing on Parallel CPU/GPU Architectures. Proceedings of the VLDB Endowment. 2013; 6(12):1374-1377. (In Eng.) doi: https://doi.org/10.14778/2536274.2536319
2. Appuswamy R., Karpathiotakis M., Porobic D., Ailamaki A. The Case For Heterogeneous HTAP. Proceedings of the 8th Biennial Conference on Innovative Data Systems Research (CIDR'17). Chaminade, California, USA; 2017. Available at: https://www.cidrdb.org/cidr2017/papers/p21-appuswamy-cidr17.pdf (accessed 13.10.2021). (In Eng.)
3. Ding B., Das S., Wu W., Chaudhuri S., Narasayya V. Plan Stitch: Harnessing the Best of Many Plans. Proceedings of the VLDB Endowment. 2018; 11(10):1123-1136. (In Eng.) doi: https://doi.org/10.14778/3231751.3231761
4. Giannikis G., Makreshanski D., Alonso G., Kossmann D. Shared Workload Optimization. Proceedings of the VLDB Endowment. 2014; 7(6):429-440. (In Eng.) doi: https://doi.org/10.14778/2732279.2732280
5. Mishra A., et al. Accelerating Analytics with Dynamic In-Memory Expressions. Proceedings of the VLDB Endowment. 2016; 9(13):1437-1448. (In Eng.) doi: https://doi.org/10.14778/3007263.3007280
6. Barthels C., Müller I., Schneider T., Alonso G., Hoefler T. Distributed Join Algorithms on Thousands of Cores. Proceedings of the VLDB Endowment. 2017; 10(5):517-528. (In Eng.) doi: https://doi.org/10.14778/3055540.3055545
7. Graefe G. Encapsulation of Parallelism in the Volcano Query Processing System. Proceedings of the 1990 ACM SIGMOD International Conference on Management of Data (Atlantic City, New Jersey, USA) (SIGMOD'90). Association for Computing Machinery, New York, NY, USA; 199. p. 102-111. (In Eng.) doi: https://doi.org/10.1145/93597.98720
8. Shaikhha A., Klonatos Ya., Parreaux L., Brown L., Dashti M., Koch C. How to Architect a Query Compiler. Proceedings of the 2016 International Conference on Management of Data (San Francisco, California, USA) (SIGMOD'16). Association for Computing Machinery, New York, NY, USA; 2016. p. 1907-1922. (In Eng.) doi: https://doi.org/10.1145/2882903.2915244
9. DeWitt D.J., Katz R.H., Olken F., Shapiro L.D., Stonebraker M.R., Wood D.A. Implementation Techniques for Main Memory Database Systems. ACM SIGMOD Record. 1984; 14(2):1-8. (In Eng.) doi: https://doi.org/10.1145/971697.602261
10. Stonebraker M., Cetintemel U. "One Size Fits All": An Idea Whose Time Has Come and Gone. Proceedings of the 21st International Conference on Data Engineering (ICDE'05). IEEE Computer Society, USA; 2005. p. 2-11. (In Eng.) doi: https://doi.org/10.1109/ICDE.2005.1
11. Ailamaki A., DeWitt D.J., Hill M.D., Wood D.A. DBMSs on a Modern Processor: Where Does Time Go? Proceedings of the 25th International Conference on Very Large Data Bases (VLDB'99). Morgan Kaufmann Publishers Inc., San Francisco, CA, USA; 1999. p. 266-277. Available at: https://www.vldb.org/conf/1999/P28.pdf (accessed 13.10.2021). (In Eng.)
12. Kester M.S., Athanassoulis M., Idreos S. Access Path Selection in Main-Memory Optimized Data Systems: Should I Scan or Should I Probe? Proceedings of the 2017 ACM International Conference on Management of Data (Chicago, Illinois, USA) (SIGMOD'17). Association for Computing Machinery, New York, NY, USA; 2017. p. 715-730. (In Eng.) doi: https://doi.org/10.1145/3035918.3064049
13. Neumann T. Efficiently Compiling Efficient Query Plans for Modern Hardware. Proceedings of the VLDB Endowment. 2011; 4(9):539-550. (In Eng.) doi: https://doi.org/10.14778/2002938.2002940
14. Kersten T., Leis V., Kemper A., Neumann T., Pavlo A., Boncz P. Everything You Always Wanted to Know about Compiled and Vectorized Queries but Were Afraid to Ask. Proceedings of the VLDB Endowment. 2018; 11(13):2209-2222. (In Eng.) doi: https://doi.org/10.14778/3275366.3284966
15. Lottarini A., Ramirez A., Coburn J., Kim M.A., Ranganathan P., Stodolsky D., Wachsler M. Vbench: Benchmarking Video Transcoding in the Cloud. ACM SIGPLAN Notices. 2018; 53(2):797-809. (In Eng.) doi: https://doi.org/10.1145/3296957.3173207
16. Kara K., Giceva J., Alonso G. FPGA-Based Data Partitioning. Proceedings of the 2017 ACM International Conference on Management of Data (Chicago, Illinois, USA) (SIGMOD'17). Association for Computing Machinery, New York, NY, USA; 2017. p. 433-445. (In Eng.) doi: https://doi.org/10.1145/3035918.3035946
17. Karnagel T., Habich D., Schlegel B., Lehner W. Heterogeneity-Aware Operator Placement in Column-Store DBMS. Datenbank-Spektrum. 2014; 14(3):211-221. (In Eng.) doi: https://doi.org/10.1007/s13222-014-0167-9
18. Müller M., Leich T., Pionteck T., Saake G., Teubner J., Spinczyk O. He..ro DB: A Concept for Parallel Data Processing on Heterogeneous Hardware. In: Brinkmann A., Karl W., Lankes S., Tomforde S., Pionteck T., Trinitis C. (eds.) Architecture of Computing Systems – ARCS 2020. ARCS 2020. Lecture Notes in Computer Science. Vol. 12155. Springer, Cham; 2020. p. 82096. (In Eng.) doi: https://doi.org/10.1007/978-3-030-52794-5_7
19. Zhang Ya., Zhang Yu, Lu J., Wang Sh., Liu Z., Han R. One size does not fit all: accelerating OLAP workloads with GPUs. Distributed and Parallel Databases. 2020; 38(4):995-1037. (In Eng.) doi: https://doi.org/10.1007/s10619-020-07304-z
20. Barroso L., Marty M., Patterson D., Ranganathan P. Attack of the Killer Microseconds. Communications of the ACM. 2017; 60(4):48-54. (In Eng.) doi: https://doi.org/10.1145/3015146
21. Chernishev G.A., Galaktionov V.A., Grigorev V.D., Klyuchikov E.S., Smirnov K.K. PosDB: An Architecture Overview. Programming and Computer Software. 2018; 44(1):62-74. (In Eng.) doi: https://doi.org/10.1134/S0361768818010024
22. Funke H., Breß S., Noll S., Markl V., Teubner J. Pipelined Query Processing in Coprocessor Environments. Proceedings of the 2018 International Conference on Management of Data (Houston, TX, USA) (SIGMOD'18). Association for Computing Machinery, New York, NY, USA; 2018. p. 1603-1618. (In Eng.) doi: https://doi.org/10.1145/3183713.3183734
23. Răducanu B., Boncz P., Zukowski M. Micro Adaptivity in Vectorwise. Proceedings of the 2013 ACM SIGMOD International Conference on Management of Data (New York, New York, USA) (SIGMOD'13). Association for Computing Machinery, New York, NY, USA; 2013. p. 1231-1242. (In Eng.) doi: https://doi.org/10.1145/2463676.2465292
24. Leis V., Boncz P., Kemper A., Neumann T. Morsel-Driven Parallelism: A NUMA-Aware Query Evaluation Framework for the Many-Core Age. Proceedings of the 2014 ACM SIGMOD International Conference on Management of Data (Snowbird, Utah, USA) (SIGMOD'14). Association for Computing Machinery, New York, NY, USA; 2014. p. 743-754. (In Eng.) doi: https://doi.org/10.1145/2588555.2610507
25. Diaconu C., Freedman C., Ismert E., Larson P.-A., Mittal P., Stonecipher R., Verma N., Zwilling M. Hekaton: SQL Server’s Memory-Optimized OLTP Engine. Proceedings of the 2013 ACM SIGMOD International Conference on Management of Data (New York, New York, USA) (SIGMOD'13). Association for Computing Machinery, New York, NY, USA; 2013. p. 1243-1254. (In Eng.) doi: https://doi.org/10.1145/2463676.2463710
26. Chen J., Jindel S., Walzer R., Sen R., Jimsheleishvilli N., Andrews M. The MemSQL Query Optimizer: A Modern Optimizer for Real-Time Analytics in a Distributed Database. Proceedings of the VLDB Endowment. 2016; 9(13):1401-1412. (In Eng.) doi: https://doi.org/10.14778/3007263.3007277
27. Boncz P., Zukowski M., Nes N. MonetDB/X100: Hyper-Pipelining Query Execution. Proceedings of the Second Biennial Conference on Innovative Data Systems Research (CIDR'05). Asilomar, CA, USA; 2005. Available at: http://cidrdb.org/cidr2005/papers/P19.pdf (accessed 13.10.2021). (In Eng.)
28. Psaroudakis I., Scheuer T., May N., Sellami A., Ailamaki A. Scaling up Concurrent Main-Memory Column-Store Scans: Towards Adaptive NUMA-Aware Data and Task Placement. Proceedings of the VLDB Endowment. 2015; 8(12):1442-1453. (In Eng.) doi: https://doi.org/10.14778/2824032.2824043
29. Graefe G. The Cascades Framework for Query Optimization. IEEE Data Engineering Bulletin. 1995; 18(3):19-29. Available at: https://www.cse.iitb.ac.in/infolab/Data/Courses/CS632/Papers/Cascades-graefe.pdf (accessed 13.10.2021). (In Eng.)
30. Leis V., Gubichev A., Mirchev A., Boncz P., Kemper A., Neumann T. How Good Are Query Optimizers, Really? Proceedings of the VLDB Endowment. 2015; 9(3):204-215. (In Eng.) doi: https://doi.org/10.14778/2850583.2850594
31. Cormode G., Garofalakis M., Haas P.J., Jermaine C. Synopses for Massive Data: Samples, Histograms, Wavelets, Sketches. Foundations and Trends in Databases. 2012; 4(1-3):1-294. (In Eng.) doi: https://doi.org/10.1561/1900000004
32. Sun J., Li G. An End-to-End Learning-Based Cost Estimator. Proceedings of the VLDB Endowment. 2 019; 13(3):307-319. (In Eng.) doi: https://doi.org/10.14778/3368289.3368296
33. Marcus R., Negi P., Mao H., Zhang C., Alizadeh M., Kraska T., Papaemmanouil O., Tatbul N. Neo: A Learned Query Optimizer. Proceedings of the VLDB Endowment. 2019; 12(11):1705-1718. (In Eng.) doi: https://doi.org/10.14778/3342263.3342644
34. Negi P., Interlandi M., Marcus R., Alizadeh M., Kraska T., Friedman M., Jindal A. Steering Query Optimizers: A Practical Take on Big Data Workloads. Proceedings of the 2021 International Conference on Management of Data (Virtual Event, China) (SIGMOD/PODS'21). Association for Computing Machinery, New York, NY, USA; 2021. p. 2557-2569. (In Eng.) doi: https://doi.org/10.1145/3448016.3457568
35. Horng J.-T., Kao C.-Y., Liu B.-J. A genetic algorithm for database query optimization. Proceedings of the First IEEE Conference on Evolutionary Computation. IEEE World Congress on Computational Intelligence. Vol. 1. IEEE Computer Society; 1994. p. 350-355. (In Eng.) doi: https://doi.org/10.1109/ICEC.1994.349926
36. Lutz C., Breß S., Zeuch S., Rabl T., Markl V. Pump Up the Volume: Processing Large Data on GPUs with Fast Interconnects. Proceedings of the 2020 ACM SIGMOD International Conference on Management of Data (Portland, OR, USA) (SIGMOD'20). Association for Computing Machinery, New York, NY, USA; 2020. p. 1633-1649. (In Eng.) doi: https://doi.org/10.1145/3318464.3389705
37. Shanbhag A., Madden S., Yu X. A Study of the Fundamental Performance Characteristics of GPUs and CPUs for Database Analytics. Proceedings of the 2020 ACM SIGMOD International Conference on Management of Data (Portland, OR, USA) (SIGMOD'20). Association for Computing Machinery, New York, NY, USA; 2020. p. 1617-1632. (In Eng.) doi: https://doi.org/10.1145/3318464.3380595
38. He B., Lu M., Yang K., Fang R., Govindaraju N.K., Luo Q., Sander P.V. Relational Query Coprocessing on Graphics Processors. ACM Transactions on Database Systems. 2009; 34(4):21. (In Eng.) doi: https://doi.org/10.1145/1620585.1620588
39. Heimel M., Saecker M., Pirk H., Manegold S., Markl V. Hardware-Oblivious Parallelism for in-Memory Column-Stores. Proceedings of the VLDB Endowment. 2013; 6(9):709-720. (In Eng.) doi: https://doi.org/10.14778/2536360.2536370
40. Stone J.E., Gohara D., Shi G. OpenCL: A Parallel Programming Standard for Heterogeneous Computing Systems. Computing in Science Engineering. 2010; 12(3):66-73. (In Eng.) doi: https://doi.org/10.1109/MCSE.2010.69
41. Breß S., Köcher B., Funke H., Zeuch S., Rabl T., Markl V. Generating Custom Code for Efficient Query Execution on Heterogeneous Processors. The VLDB Journal. 2018; 27(6):797-822. (In Eng.) doi: https://doi.org/10.1007/s00778-018-0512-y
42. Kohn A., Leis V., Neumann T. Making Compiling Query Engines Practical. IEEE Transactions on Knowledge and Data Engineering. 2021; 33(2):597-612. (In Eng.) doi: https://doi.org/10.1109/TKDE.2019.2905235
43. Pirk H., Moll O., Zaharia M., Madden S. Voodoo ‒ a Vector Algebra for Portable Database Performance on Modern Hardware. Proceedings of the VLDB Endowment. 2016; 9(14):1707-1718. (In Eng.) doi: https://doi.org/10.14778/3007328.3007336
44. Shaikhha A., Klonatos Ya., Koch C. Building Efficient Query Engines in a High-Level Language. ACM Transactions on Database Systems. 2018; 43(1):4. (In Eng.) doi: https://doi.org/10.1145/3183653
45. Chrysogelos P., Karpathiotakis M., Appuswamy R., Ailamaki A. HetExchange: Encapsulating Heterogeneous CPU-GPU Parallelism in JIT Compiled Engines. Proceedings of the VLDB Endowment. 2019; 12(5):544-556. (In Eng.) doi: https://doi.org/10.14778/3303753.3303760
46. Gubner T., Boncz P. Charting the Design Space of Query Execution Using VOILA. Proceedings of the VLDB Endowment. 2021; 14(6):1067-1079. (In Eng.) doi: https://doi.org/10.14778/3447689.3447709
47. Palkar S., Thomas J.J., Shanbhag A., Narayanan D., Pirk H., Schwarzkopf M., Amarasinghe S., Zaharia M. Weld: A Common Runtime for High Performance Data Analytics. Proceedings of the 8th Biennial Conference on Innovative Data Systems Research (CIDR'17). Chaminade, California, USA; 2017. Available at: https://www.cidrdb.org/cidr2017/papers/p127-palkar-cidr17.pdf (accessed 13.10.2021). (In Eng.)
48. Zhang S., He J., He B., Lu M. OmniDB: Towards Portable and Efficient Query Processing on Parallel CPU/GPU Architectures. Proceedings of the VLDB Endowment. 2013; 6(12):1374-1377. (In Eng.) doi: https://doi.org/10.14778/2536274.2536319
Опубликована
2021-12-20
Как цитировать
KURAPOV, Petr Alexandrovich; MELIK-ADAMYAN, Areg Frikovich.
Оптимизация аналитических запросов в гетерогенных системах.
Современные информационные технологии и ИТ-образование, [S.l.], v. 17, n. 4, p. 972-987, dec. 2021.
ISSN 2411-1473. Доступно на: <http://sitito.cs.msu.ru/index.php/SITITO/article/view/801>. Дата доступа: 10 july 2025
doi: https://doi.org/10.25559/SITITO.17.202104.972-987.
Раздел
Исследования и разработки в области новых ИТ и их приложений
Это произведение доступно по лицензии Creative Commons «Attribution» («Атрибуция») 4.0 Всемирная.
Редакционная политика журнала основывается на традиционных этических принципах российской научной периодики и строится с учетом этических норм работы редакторов и издателей, закрепленных в Кодексе поведения и руководящих принципах наилучшей практики для редактора журнала (Code of Conduct and Best Practice Guidelines for Journal Editors) и Кодексе поведения для издателя журнала (Code of Conduct for Journal Publishers), разработанных Комитетом по публикационной этике - Committee on Publication Ethics (COPE). В процессе издательской деятельности редколлегия журнала руководствуется международными правилами охраны авторского права, нормами действующего законодательства РФ, международными издательскими стандартами и обязательной ссылке на первоисточник.
Журнал позволяет авторам сохранять авторское право без ограничений. Журнал позволяет авторам сохранить права на публикацию без ограничений.
Издательская политика в области авторского права и архивирования определяются «зеленым цветом» в базе данных SHERPA/RoMEO.
Все статьи распространяются на условиях лицензии Creative Commons «Attribution» («Атрибуция») 4.0 Всемирная, которая позволяет другим использовать, распространять, дополнять эту работу с обязательной ссылкой на оригинальную работу и публикацию в этом журналe.
