/
githubmirror
/
spark
Обзор
Документация
Войти
/
githubmirror
/
spark
Код
Запросы
0
Пакеты
0
Релизы
0
Аналитика
Безопасность
master
sql/core/benchmarks/DataSourceReadBenchmark-jdk25-results.txt
436 строк
43 KB
Dongjoon Hyun
[SPARK-58415][INFRA][TEST] Regenerate benchmark results
29 июл 2026, 10:51
29 июл 2026, 10:51
6b793ce
Код
Авторство
О чём код?
================================================================================================ SQL Single Numeric Column Scan ================================================================================================ OpenJDK 64-Bit Server VM 25.0.4+7-LTS on Linux 6.17.0-1020-azure AMD EPYC 9V74 80-Core Processor SQL Single BOOLEAN Column Scan: Best Time(ms) Avg Time(ms) Stdev(ms) Rate(M/s) Per Row(ns) Relative ------------------------------------------------------------------------------------------------------------------------ SQL CSV 9489 9542 74 1.7 603.3 1.0X SQL Json 6332 6366 48 2.5 402.6 1.5X SQL Json with UnsafeRow 6990 7000 14 2.3 444.4 1.4X SQL Parquet Vectorized: DataPageV1 80 92 9 197.1 5.1 118.9X SQL Parquet Vectorized: DataPageV2 91 97 6 172.2 5.8 103.9X SQL Parquet MR: DataPageV1 1793 1802 12 8.8 114.0 5.3X SQL Parquet MR: DataPageV2 1699 1711 16 9.3 108.1 5.6X SQL ORC Vectorized 140 153 5 112.3 8.9 67.8X SQL ORC MR 1572 1575 4 10.0 99.9 6.0X OpenJDK 64-Bit Server VM 25.0.4+7-LTS on Linux 6.17.0-1020-azure AMD EPYC 9V74 80-Core Processor Parquet Reader Single BOOLEAN Column Scan: Best Time(ms) Avg Time(ms) Stdev(ms) Rate(M/s) Per Row(ns) Relative --------------------------------------------------------------------------------------------------------------------------- ParquetReader Vectorized: DataPageV1 30 32 2 524.8 1.9 1.0X ParquetReader Vectorized: DataPageV2 48 50 2 328.9 3.0 0.6X ParquetReader Vectorized -> Row: DataPageV1 29 31 3 538.8 1.9 1.0X ParquetReader Vectorized -> Row: DataPageV2 46 48 2 338.3 3.0 0.6X OpenJDK 64-Bit Server VM 25.0.4+7-LTS on Linux 6.17.0-1020-azure AMD EPYC 9V74 80-Core Processor SQL Single TINYINT Column Scan: Best Time(ms) Avg Time(ms) Stdev(ms) Rate(M/s) Per Row(ns) Relative ------------------------------------------------------------------------------------------------------------------------ SQL CSV 9453 9545 130 1.7 601.0 1.0X SQL Json 7611 7669 82 2.1 483.9 1.2X SQL Json with UnsafeRow 8151 8158 9 1.9 518.2 1.2X SQL Parquet Vectorized: DataPageV1 102 110 7 154.0 6.5 92.6X SQL Parquet Vectorized: DataPageV2 101 114 8 155.0 6.4 93.2X SQL Parquet MR: DataPageV1 1945 1955 15 8.1 123.6 4.9X SQL Parquet MR: DataPageV2 1884 1902 25 8.3 119.8 5.0X SQL ORC Vectorized 116 125 10 136.0 7.4 81.8X SQL ORC MR 1634 1650 23 9.6 103.9 5.8X OpenJDK 64-Bit Server VM 25.0.4+7-LTS on Linux 6.17.0-1020-azure AMD EPYC 9V74 80-Core Processor Parquet Reader Single TINYINT Column Scan: Best Time(ms) Avg Time(ms) Stdev(ms) Rate(M/s) Per Row(ns) Relative --------------------------------------------------------------------------------------------------------------------------- ParquetReader Vectorized: DataPageV1 42 43 1 373.2 2.7 1.0X ParquetReader Vectorized: DataPageV2 42 44 2 373.1 2.7 1.0X ParquetReader Vectorized -> Row: DataPageV1 43 43 1 366.5 2.7 1.0X ParquetReader Vectorized -> Row: DataPageV2 43 44 1 364.0 2.7 1.0X OpenJDK 64-Bit Server VM 25.0.4+7-LTS on Linux 6.17.0-1020-azure AMD EPYC 9V74 80-Core Processor SQL Single SMALLINT Column Scan: Best Time(ms) Avg Time(ms) Stdev(ms) Rate(M/s) Per Row(ns) Relative ------------------------------------------------------------------------------------------------------------------------ SQL CSV 10173 10174 1 1.5 646.8 1.0X SQL Json 7824 7832 12 2.0 497.4 1.3X SQL Json with UnsafeRow 8540 8560 29 1.8 543.0 1.2X SQL Parquet Vectorized: DataPageV1 119 129 7 131.9 7.6 85.3X SQL Parquet Vectorized: DataPageV2 162 167 3 97.3 10.3 62.9X SQL Parquet MR: DataPageV1 2079 2127 68 7.6 132.2 4.9X SQL Parquet MR: DataPageV2 1977 1991 19 8.0 125.7 5.1X SQL ORC Vectorized 150 155 3 104.7 9.6 67.7X SQL ORC MR 1716 1720 7 9.2 109.1 5.9X OpenJDK 64-Bit Server VM 25.0.4+7-LTS on Linux 6.17.0-1020-azure AMD EPYC 9V74 80-Core Processor Parquet Reader Single SMALLINT Column Scan: Best Time(ms) Avg Time(ms) Stdev(ms) Rate(M/s) Per Row(ns) Relative --------------------------------------------------------------------------------------------------------------------------- ParquetReader Vectorized: DataPageV1 146 147 1 107.9 9.3 1.0X ParquetReader Vectorized: DataPageV2 182 186 5 86.6 11.5 0.8X ParquetReader Vectorized -> Row: DataPageV1 150 151 2 104.9 9.5 1.0X ParquetReader Vectorized -> Row: DataPageV2 185 185 0 85.2 11.7 0.8X OpenJDK 64-Bit Server VM 25.0.4+7-LTS on Linux 6.17.0-1020-azure AMD EPYC 9V74 80-Core Processor SQL Single INT Column Scan: Best Time(ms) Avg Time(ms) Stdev(ms) Rate(M/s) Per Row(ns) Relative ------------------------------------------------------------------------------------------------------------------------ SQL CSV 11296 11302 9 1.4 718.2 1.0X SQL Json 8372 8445 103 1.9 532.3 1.3X SQL Json with UnsafeRow 9196 9207 14 1.7 584.7 1.2X SQL Parquet Vectorized: DataPageV1 114 122 8 138.2 7.2 99.2X SQL Parquet Vectorized: DataPageV2 199 200 2 79.1 12.6 56.8X SQL Parquet MR: DataPageV1 2103 2104 1 7.5 133.7 5.4X SQL Parquet MR: DataPageV2 1969 1974 7 8.0 125.2 5.7X SQL ORC Vectorized 187 194 6 84.1 11.9 60.4X SQL ORC MR 1766 1766 0 8.9 112.3 6.4X OpenJDK 64-Bit Server VM 25.0.4+7-LTS on Linux 6.17.0-1020-azure AMD EPYC 9V74 80-Core Processor Parquet Reader Single INT Column Scan: Best Time(ms) Avg Time(ms) Stdev(ms) Rate(M/s) Per Row(ns) Relative --------------------------------------------------------------------------------------------------------------------------- ParquetReader Vectorized: DataPageV1 135 137 1 116.3 8.6 1.0X ParquetReader Vectorized: DataPageV2 203 206 2 77.5 12.9 0.7X ParquetReader Vectorized -> Row: DataPageV1 137 141 2 114.9 8.7 1.0X ParquetReader Vectorized -> Row: DataPageV2 203 205 2 77.6 12.9 0.7X OpenJDK 64-Bit Server VM 25.0.4+7-LTS on Linux 6.17.0-1020-azure AMD EPYC 9V74 80-Core Processor SQL Single BIGINT Column Scan: Best Time(ms) Avg Time(ms) Stdev(ms) Rate(M/s) Per Row(ns) Relative ------------------------------------------------------------------------------------------------------------------------ SQL CSV 11271 11284 18 1.4 716.6 1.0X SQL Json 8360 8377 25 1.9 531.5 1.3X SQL Json with UnsafeRow 9163 9276 161 1.7 582.5 1.2X SQL Parquet Vectorized: DataPageV1 303 313 5 51.9 19.3 37.2X SQL Parquet Vectorized: DataPageV2 172 176 3 91.2 11.0 65.4X SQL Parquet MR: DataPageV1 2610 2625 21 6.0 166.0 4.3X SQL Parquet MR: DataPageV2 2024 2034 14 7.8 128.7 5.6X SQL ORC Vectorized 167 176 6 94.2 10.6 67.5X SQL ORC MR 1847 1858 16 8.5 117.4 6.1X OpenJDK 64-Bit Server VM 25.0.4+7-LTS on Linux 6.17.0-1020-azure AMD EPYC 9V74 80-Core Processor Parquet Reader Single BIGINT Column Scan: Best Time(ms) Avg Time(ms) Stdev(ms) Rate(M/s) Per Row(ns) Relative --------------------------------------------------------------------------------------------------------------------------- ParquetReader Vectorized: DataPageV1 368 384 11 42.8 23.4 1.0X ParquetReader Vectorized: DataPageV2 227 230 4 69.2 14.4 1.6X ParquetReader Vectorized -> Row: DataPageV1 370 372 3 42.5 23.5 1.0X ParquetReader Vectorized -> Row: DataPageV2 227 232 3 69.3 14.4 1.6X OpenJDK 64-Bit Server VM 25.0.4+7-LTS on Linux 6.17.0-1020-azure AMD EPYC 9V74 80-Core Processor SQL Single FLOAT Column Scan: Best Time(ms) Avg Time(ms) Stdev(ms) Rate(M/s) Per Row(ns) Relative ------------------------------------------------------------------------------------------------------------------------ SQL CSV 11402 11410 12 1.4 724.9 1.0X SQL Json 10857 10900 60 1.4 690.3 1.1X SQL Json with UnsafeRow 11661 11671 14 1.3 741.4 1.0X SQL Parquet Vectorized: DataPageV1 83 93 5 189.2 5.3 137.1X SQL Parquet Vectorized: DataPageV2 81 88 7 194.7 5.1 141.1X SQL Parquet MR: DataPageV1 2037 2041 6 7.7 129.5 5.6X SQL Parquet MR: DataPageV2 2026 2029 4 7.8 128.8 5.6X SQL ORC Vectorized 241 255 14 65.3 15.3 47.3X SQL ORC MR 1833 1833 0 8.6 116.5 6.2X OpenJDK 64-Bit Server VM 25.0.4+7-LTS on Linux 6.17.0-1020-azure AMD EPYC 9V74 80-Core Processor Parquet Reader Single FLOAT Column Scan: Best Time(ms) Avg Time(ms) Stdev(ms) Rate(M/s) Per Row(ns) Relative --------------------------------------------------------------------------------------------------------------------------- ParquetReader Vectorized: DataPageV1 151 152 1 104.4 9.6 1.0X ParquetReader Vectorized: DataPageV2 151 153 1 104.1 9.6 1.0X ParquetReader Vectorized -> Row: DataPageV1 158 165 9 99.5 10.1 1.0X ParquetReader Vectorized -> Row: DataPageV2 154 156 2 101.9 9.8 1.0X OpenJDK 64-Bit Server VM 25.0.4+7-LTS on Linux 6.17.0-1020-azure AMD EPYC 9V74 80-Core Processor SQL Single DOUBLE Column Scan: Best Time(ms) Avg Time(ms) Stdev(ms) Rate(M/s) Per Row(ns) Relative ------------------------------------------------------------------------------------------------------------------------ SQL CSV 11965 11994 41 1.3 760.7 1.0X SQL Json 11404 11415 15 1.4 725.1 1.0X SQL Json with UnsafeRow 12100 12114 20 1.3 769.3 1.0X SQL Parquet Vectorized: DataPageV1 307 310 3 51.3 19.5 39.0X SQL Parquet Vectorized: DataPageV2 303 305 2 52.0 19.2 39.6X SQL Parquet MR: DataPageV1 2503 2510 9 6.3 159.2 4.8X SQL Parquet MR: DataPageV2 2438 2442 5 6.5 155.0 4.9X SQL ORC Vectorized 484 496 16 32.5 30.8 24.7X SQL ORC MR 2097 2106 13 7.5 133.3 5.7X OpenJDK 64-Bit Server VM 25.0.4+7-LTS on Linux 6.17.0-1020-azure AMD EPYC 9V74 80-Core Processor Parquet Reader Single DOUBLE Column Scan: Best Time(ms) Avg Time(ms) Stdev(ms) Rate(M/s) Per Row(ns) Relative --------------------------------------------------------------------------------------------------------------------------- ParquetReader Vectorized: DataPageV1 383 387 4 41.1 24.3 1.0X ParquetReader Vectorized: DataPageV2 366 379 7 42.9 23.3 1.0X ParquetReader Vectorized -> Row: DataPageV1 373 375 1 42.2 23.7 1.0X ParquetReader Vectorized -> Row: DataPageV2 376 387 7 41.8 23.9 1.0X ================================================================================================ SQL Single Numeric Column Scan in Struct ================================================================================================ OpenJDK 64-Bit Server VM 25.0.4+7-LTS on Linux 6.17.0-1020-azure AMD EPYC 9V74 80-Core Processor SQL Single TINYINT Column Scan in Struct: Best Time(ms) Avg Time(ms) Stdev(ms) Rate(M/s) Per Row(ns) Relative ------------------------------------------------------------------------------------------------------------------------------------------- SQL ORC MR 2237 2238 2 7.0 142.2 1.0X SQL ORC Vectorized (Nested Column Disabled) 2180 2186 9 7.2 138.6 1.0X SQL ORC Vectorized (Nested Column Enabled) 132 139 10 119.1 8.4 16.9X SQL Parquet MR: DataPageV1 2278 2317 56 6.9 144.8 1.0X SQL Parquet Vectorized: DataPageV1 (Nested Column Disabled) 2678 2681 3 5.9 170.3 0.8X SQL Parquet Vectorized: DataPageV1 (Nested Column Enabled) 99 104 5 158.1 6.3 22.5X SQL Parquet MR: DataPageV2 2308 2321 17 6.8 146.8 1.0X SQL Parquet Vectorized: DataPageV2 (Nested Column Disabled) 2623 2651 39 6.0 166.8 0.9X SQL Parquet Vectorized: DataPageV2 (Nested Column Enabled) 99 103 6 158.5 6.3 22.5X OpenJDK 64-Bit Server VM 25.0.4+7-LTS on Linux 6.17.0-1020-azure AMD EPYC 9V74 80-Core Processor SQL Single SMALLINT Column Scan in Struct: Best Time(ms) Avg Time(ms) Stdev(ms) Rate(M/s) Per Row(ns) Relative ------------------------------------------------------------------------------------------------------------------------------------------- SQL ORC MR 2198 2206 11 7.2 139.8 1.0X SQL ORC Vectorized (Nested Column Disabled) 2217 2226 13 7.1 140.9 1.0X SQL ORC Vectorized (Nested Column Enabled) 157 166 7 99.9 10.0 14.0X SQL Parquet MR: DataPageV1 2393 2414 30 6.6 152.1 0.9X SQL Parquet Vectorized: DataPageV1 (Nested Column Disabled) 2730 2754 35 5.8 173.5 0.8X SQL Parquet Vectorized: DataPageV1 (Nested Column Enabled) 133 138 4 118.3 8.5 16.5X SQL Parquet MR: DataPageV2 2366 2384 25 6.6 150.4 0.9X SQL Parquet Vectorized: DataPageV2 (Nested Column Disabled) 2681 2710 40 5.9 170.5 0.8X SQL Parquet Vectorized: DataPageV2 (Nested Column Enabled) 157 167 7 100.0 10.0 14.0X OpenJDK 64-Bit Server VM 25.0.4+7-LTS on Linux 6.17.0-1020-azure AMD EPYC 9V74 80-Core Processor SQL Single INT Column Scan in Struct: Best Time(ms) Avg Time(ms) Stdev(ms) Rate(M/s) Per Row(ns) Relative ------------------------------------------------------------------------------------------------------------------------------------------- SQL ORC MR 2285 2318 46 6.9 145.3 1.0X SQL ORC Vectorized (Nested Column Disabled) 2237 2246 13 7.0 142.2 1.0X SQL ORC Vectorized (Nested Column Enabled) 145 163 23 108.8 9.2 15.8X SQL Parquet MR: DataPageV1 2487 2497 14 6.3 158.1 0.9X SQL Parquet Vectorized: DataPageV1 (Nested Column Disabled) 2933 2938 7 5.4 186.5 0.8X SQL Parquet Vectorized: DataPageV1 (Nested Column Enabled) 114 120 7 138.4 7.2 20.1X SQL Parquet MR: DataPageV2 2402 2405 4 6.5 152.7 1.0X SQL Parquet Vectorized: DataPageV2 (Nested Column Disabled) 2833 2840 9 5.6 180.1 0.8X SQL Parquet Vectorized: DataPageV2 (Nested Column Enabled) 179 186 7 87.8 11.4 12.8X OpenJDK 64-Bit Server VM 25.0.4+7-LTS on Linux 6.17.0-1020-azure AMD EPYC 9V74 80-Core Processor SQL Single BIGINT Column Scan in Struct: Best Time(ms) Avg Time(ms) Stdev(ms) Rate(M/s) Per Row(ns) Relative ------------------------------------------------------------------------------------------------------------------------------------------- SQL ORC MR 2265 2273 10 6.9 144.0 1.0X SQL ORC Vectorized (Nested Column Disabled) 2282 2287 7 6.9 145.1 1.0X SQL ORC Vectorized (Nested Column Enabled) 142 151 10 110.8 9.0 16.0X SQL Parquet MR: DataPageV1 2915 2925 14 5.4 185.3 0.8X SQL Parquet Vectorized: DataPageV1 (Nested Column Disabled) 3468 3476 12 4.5 220.5 0.7X SQL Parquet Vectorized: DataPageV1 (Nested Column Enabled) 318 327 5 49.4 20.2 7.1X SQL Parquet MR: DataPageV2 2364 2372 12 6.7 150.3 1.0X SQL Parquet Vectorized: DataPageV2 (Nested Column Disabled) 2725 2744 27 5.8 173.3 0.8X SQL Parquet Vectorized: DataPageV2 (Nested Column Enabled) 168 178 7 93.4 10.7 13.5X OpenJDK 64-Bit Server VM 25.0.4+7-LTS on Linux 6.17.0-1020-azure AMD EPYC 9V74 80-Core Processor SQL Single FLOAT Column Scan in Struct: Best Time(ms) Avg Time(ms) Stdev(ms) Rate(M/s) Per Row(ns) Relative ------------------------------------------------------------------------------------------------------------------------------------------- SQL ORC MR 2382 2383 1 6.6 151.5 1.0X SQL ORC Vectorized (Nested Column Disabled) 2376 2392 22 6.6 151.1 1.0X SQL ORC Vectorized (Nested Column Enabled) 236 244 7 66.6 15.0 10.1X SQL Parquet MR: DataPageV1 2441 2480 56 6.4 155.2 1.0X SQL Parquet Vectorized: DataPageV1 (Nested Column Disabled) 2878 2897 27 5.5 183.0 0.8X SQL Parquet Vectorized: DataPageV1 (Nested Column Enabled) 86 92 7 183.1 5.5 27.7X SQL Parquet MR: DataPageV2 2419 2421 2 6.5 153.8 1.0X SQL Parquet Vectorized: DataPageV2 (Nested Column Disabled) 2801 2803 3 5.6 178.1 0.9X SQL Parquet Vectorized: DataPageV2 (Nested Column Enabled) 86 94 9 183.5 5.5 27.8X OpenJDK 64-Bit Server VM 25.0.4+7-LTS on Linux 6.17.0-1020-azure AMD EPYC 9V74 80-Core Processor SQL Single DOUBLE Column Scan in Struct: Best Time(ms) Avg Time(ms) Stdev(ms) Rate(M/s) Per Row(ns) Relative ------------------------------------------------------------------------------------------------------------------------------------------- SQL ORC MR 2602 2666 91 6.0 165.4 1.0X SQL ORC Vectorized (Nested Column Disabled) 2620 2646 38 6.0 166.6 1.0X SQL ORC Vectorized (Nested Column Enabled) 484 491 6 32.5 30.8 5.4X SQL Parquet MR: DataPageV1 2948 2949 2 5.3 187.4 0.9X SQL Parquet Vectorized: DataPageV1 (Nested Column Disabled) 3417 3429 17 4.6 217.2 0.8X SQL Parquet Vectorized: DataPageV1 (Nested Column Enabled) 305 318 7 51.5 19.4 8.5X SQL Parquet MR: DataPageV2 2816 2828 18 5.6 179.0 0.9X SQL Parquet Vectorized: DataPageV2 (Nested Column Disabled) 3440 3524 119 4.6 218.7 0.8X SQL Parquet Vectorized: DataPageV2 (Nested Column Enabled) 295 318 13 53.4 18.7 8.8X ================================================================================================ SQL Nested Column Scan ================================================================================================ OpenJDK 64-Bit Server VM 25.0.4+7-LTS on Linux 6.17.0-1020-azure AMD EPYC 9V74 80-Core Processor SQL Nested Column Scan: Best Time(ms) Avg Time(ms) Stdev(ms) Rate(M/s) Per Row(ns) Relative ------------------------------------------------------------------------------------------------------------------------------------------- SQL ORC MR 12603 12658 59 0.1 12019.6 1.0X SQL ORC Vectorized (Nested Column Disabled) 12670 12715 36 0.1 12083.0 1.0X SQL ORC Vectorized (Nested Column Enabled) 6868 6879 7 0.2 6549.7 1.8X SQL Parquet MR: DataPageV1 8424 8450 25 0.1 8033.7 1.5X SQL Parquet Vectorized: DataPageV1 (Nested Column Disabled) 8782 8801 17 0.1 8375.3 1.4X SQL Parquet Vectorized: DataPageV1 (Nested Column Enabled) 5424 5478 49 0.2 5172.8 2.3X SQL Parquet MR: DataPageV2 9203 9245 67 0.1 8777.0 1.4X SQL Parquet Vectorized: DataPageV2 (Nested Column Disabled) 9475 9508 28 0.1 9035.8 1.3X SQL Parquet Vectorized: DataPageV2 (Nested Column Enabled) 4964 4999 23 0.2 4734.0 2.5X ================================================================================================ Int and String Scan ================================================================================================ OpenJDK 64-Bit Server VM 25.0.4+7-LTS on Linux 6.17.0-1020-azure AMD EPYC 9V74 80-Core Processor Int and String Scan: Best Time(ms) Avg Time(ms) Stdev(ms) Rate(M/s) Per Row(ns) Relative ------------------------------------------------------------------------------------------------------------------------ SQL CSV 10787 10903 165 1.0 1028.7 1.0X SQL Json 9578 9611 47 1.1 913.4 1.1X SQL Parquet Vectorized: DataPageV1 1934 1949 22 5.4 184.4 5.6X SQL Parquet Vectorized: DataPageV2 2025 2066 58 5.2 193.1 5.3X SQL Parquet MR: DataPageV1 3811 3822 16 2.8 363.4 2.8X SQL Parquet MR: DataPageV2 3737 3785 68 2.8 356.4 2.9X SQL ORC Vectorized 1706 1709 5 6.1 162.6 6.3X SQL ORC MR 3346 3362 23 3.1 319.1 3.2X ================================================================================================ Repeated String Scan ================================================================================================ OpenJDK 64-Bit Server VM 25.0.4+7-LTS on Linux 6.17.0-1020-azure AMD EPYC 9V74 80-Core Processor Repeated String: Best Time(ms) Avg Time(ms) Stdev(ms) Rate(M/s) Per Row(ns) Relative ------------------------------------------------------------------------------------------------------------------------ SQL CSV 5621 5634 19 1.9 536.0 1.0X SQL Json 5750 5759 13 1.8 548.4 1.0X SQL Parquet Vectorized: DataPageV1 461 477 18 22.8 43.9 12.2X SQL Parquet Vectorized: DataPageV2 456 459 2 23.0 43.5 12.3X SQL Parquet MR: DataPageV1 1646 1648 2 6.4 157.0 3.4X SQL Parquet MR: DataPageV2 1633 1648 20 6.4 155.8 3.4X SQL ORC Vectorized 413 418 6 25.4 39.4 13.6X SQL ORC MR 1620 1623 4 6.5 154.5 3.5X ================================================================================================ Partitioned Table Scan ================================================================================================ OpenJDK 64-Bit Server VM 25.0.4+7-LTS on Linux 6.17.0-1020-azure AMD EPYC 9V74 80-Core Processor Partitioned Table: Best Time(ms) Avg Time(ms) Stdev(ms) Rate(M/s) Per Row(ns) Relative --------------------------------------------------------------------------------------------------------------------------------- Data column - CSV 11589 11711 173 1.4 736.8 1.0X Data column - Json 8106 8117 17 1.9 515.3 1.4X Data column - Parquet Vectorized: DataPageV1 109 118 7 144.0 6.9 106.1X Data column - Parquet Vectorized: DataPageV2 183 190 5 85.7 11.7 63.2X Data column - Parquet MR: DataPageV1 2226 2246 28 7.1 141.5 5.2X Data column - Parquet MR: DataPageV2 2130 2153 33 7.4 135.4 5.4X Data column - ORC Vectorized 162 178 11 97.1 10.3 71.6X Data column - ORC MR 1923 1960 52 8.2 122.3 6.0X Partition column - CSV 3531 3537 7 4.5 224.5 3.3X Partition column - Json 7030 7043 19 2.2 447.0 1.6X Partition column - Parquet Vectorized: DataPageV1 33 39 7 477.0 2.1 351.4X Partition column - Parquet Vectorized: DataPageV2 32 40 10 494.0 2.0 364.0X Partition column - Parquet MR: DataPageV1 1197 1200 4 13.1 76.1 9.7X Partition column - Parquet MR: DataPageV2 1185 1187 3 13.3 75.4 9.8X Partition column - ORC Vectorized 33 38 8 483.5 2.1 356.3X Partition column - ORC MR 1186 1192 8 13.3 75.4 9.8X Both columns - CSV 11748 11818 99 1.3 746.9 1.0X Both columns - Json 8337 8377 56 1.9 530.1 1.4X Both columns - Parquet Vectorized: DataPageV1 133 139 6 118.2 8.5 87.1X Both columns - Parquet Vectorized: DataPageV2 208 219 9 75.7 13.2 55.8X Both columns - Parquet MR: DataPageV1 2290 2291 2 6.9 145.6 5.1X Both columns - Parquet MR: DataPageV2 2198 2237 55 7.2 139.7 5.3X Both columns - ORC Vectorized 193 202 6 81.3 12.3 59.9X Both columns - ORC MR 1965 1980 21 8.0 124.9 5.9X ================================================================================================ String with Nulls Scan ================================================================================================ OpenJDK 64-Bit Server VM 25.0.4+7-LTS on Linux 6.17.0-1020-azure AMD EPYC 9V74 80-Core Processor String with Nulls Scan (0.0%): Best Time(ms) Avg Time(ms) Stdev(ms) Rate(M/s) Per Row(ns) Relative ------------------------------------------------------------------------------------------------------------------------ SQL CSV 6804 6830 38 1.5 648.9 1.0X SQL Json 7873 7881 11 1.3 750.8 0.9X SQL Parquet Vectorized: DataPageV1 1174 1186 16 8.9 112.0 5.8X SQL Parquet Vectorized: DataPageV2 1309 1329 28 8.0 124.9 5.2X SQL Parquet MR: DataPageV1 3406 3428 31 3.1 324.8 2.0X SQL Parquet MR: DataPageV2 3305 3318 19 3.2 315.2 2.1X ParquetReader Vectorized: DataPageV1 843 851 14 12.4 80.4 8.1X ParquetReader Vectorized: DataPageV2 925 955 42 11.3 88.2 7.4X SQL ORC Vectorized 689 716 28 15.2 65.7 9.9X SQL ORC MR 2739 2740 1 3.8 261.2 2.5X OpenJDK 64-Bit Server VM 25.0.4+7-LTS on Linux 6.17.0-1020-azure AMD EPYC 9V74 80-Core Processor String with Nulls Scan (50.0%): Best Time(ms) Avg Time(ms) Stdev(ms) Rate(M/s) Per Row(ns) Relative ------------------------------------------------------------------------------------------------------------------------ SQL CSV 4895 4896 1 2.1 466.9 1.0X SQL Json 6747 6750 5 1.6 643.4 0.7X SQL Parquet Vectorized: DataPageV1 831 842 10 12.6 79.2 5.9X SQL Parquet Vectorized: DataPageV2 972 978 6 10.8 92.7 5.0X SQL Parquet MR: DataPageV1 2569 2588 27 4.1 245.0 1.9X SQL Parquet MR: DataPageV2 2581 2594 18 4.1 246.1 1.9X ParquetReader Vectorized: DataPageV1 759 765 7 13.8 72.4 6.5X ParquetReader Vectorized: DataPageV2 882 890 13 11.9 84.1 5.6X SQL ORC Vectorized 973 981 12 10.8 92.8 5.0X SQL ORC MR 2692 2699 10 3.9 256.7 1.8X OpenJDK 64-Bit Server VM 25.0.4+7-LTS on Linux 6.17.0-1020-azure AMD EPYC 9V74 80-Core Processor String with Nulls Scan (95.0%): Best Time(ms) Avg Time(ms) Stdev(ms) Rate(M/s) Per Row(ns) Relative ------------------------------------------------------------------------------------------------------------------------ SQL CSV 3320 3322 3 3.2 316.7 1.0X SQL Json 4562 4578 23 2.3 435.0 0.7X SQL Parquet Vectorized: DataPageV1 173 177 5 60.7 16.5 19.2X SQL Parquet Vectorized: DataPageV2 191 197 9 54.9 18.2 17.4X SQL Parquet MR: DataPageV1 1684 1686 3 6.2 160.6 2.0X SQL Parquet MR: DataPageV2 1641 1643 3 6.4 156.5 2.0X ParquetReader Vectorized: DataPageV1 194 195 1 54.1 18.5 17.1X ParquetReader Vectorized: DataPageV2 211 213 1 49.6 20.1 15.7X SQL ORC Vectorized 295 297 3 35.6 28.1 11.3X SQL ORC MR 1596 1599 5 6.6 152.2 2.1X ================================================================================================ Single Column Scan From Wide Columns ================================================================================================ OpenJDK 64-Bit Server VM 25.0.4+7-LTS on Linux 6.17.0-1020-azure AMD EPYC 9V74 80-Core Processor Single Column Scan from 10 columns: Best Time(ms) Avg Time(ms) Stdev(ms) Rate(M/s) Per Row(ns) Relative ------------------------------------------------------------------------------------------------------------------------ SQL CSV 1264 1266 3 0.8 1205.5 1.0X SQL Json 1685 1685 1 0.6 1606.7 0.8X SQL Parquet Vectorized: DataPageV1 26 33 7 39.6 25.2 47.7X SQL Parquet Vectorized: DataPageV2 31 34 5 34.3 29.2 41.3X SQL Parquet MR: DataPageV1 160 164 5 6.6 152.3 7.9X SQL Parquet MR: DataPageV2 159 172 6 6.6 151.7 7.9X SQL ORC Vectorized 29 34 6 36.7 27.2 44.3X SQL ORC MR 138 145 8 7.6 131.3 9.2X OpenJDK 64-Bit Server VM 25.0.4+7-LTS on Linux 6.17.0-1020-azure AMD EPYC 9V74 80-Core Processor Single Column Scan from 50 columns: Best Time(ms) Avg Time(ms) Stdev(ms) Rate(M/s) Per Row(ns) Relative ------------------------------------------------------------------------------------------------------------------------ SQL CSV 2725 2742 24 0.4 2598.8 1.0X SQL Json 6339 6394 77 0.2 6045.7 0.4X SQL Parquet Vectorized: DataPageV1 29 33 5 36.8 27.2 95.6X SQL Parquet Vectorized: DataPageV2 33 36 5 32.1 31.1 83.5X SQL Parquet MR: DataPageV1 164 172 6 6.4 156.2 16.6X SQL Parquet MR: DataPageV2 160 173 9 6.6 152.4 17.1X SQL ORC Vectorized 31 36 6 33.3 30.0 86.6X SQL ORC MR 138 144 7 7.6 132.0 19.7X OpenJDK 64-Bit Server VM 25.0.4+7-LTS on Linux 6.17.0-1020-azure AMD EPYC 9V74 80-Core Processor Single Column Scan from 100 columns: Best Time(ms) Avg Time(ms) Stdev(ms) Rate(M/s) Per Row(ns) Relative ------------------------------------------------------------------------------------------------------------------------ SQL CSV 4522 4544 32 0.2 4312.5 1.0X SQL Json 12085 12127 61 0.1 11524.8 0.4X SQL Parquet Vectorized: DataPageV1 35 39 6 30.2 33.1 130.2X SQL Parquet Vectorized: DataPageV2 39 44 5 26.8 37.3 115.5X SQL Parquet MR: DataPageV1 174 181 6 6.0 165.8 26.0X SQL Parquet MR: DataPageV2 170 178 8 6.2 161.7 26.7X SQL ORC Vectorized 37 41 5 28.2 35.4 121.8X SQL ORC MR 144 150 5 7.3 137.1 31.5X