/
githubmirror
/
spark
Обзор
Документация
Войти
/
githubmirror
/
spark
Код
Запросы
0
Пакеты
0
Релизы
0
Аналитика
Безопасность
master
sql/core/benchmarks/DataSourceReadBenchmark-results.txt
436 строк
43 KB
Dongjoon Hyun
[SPARK-58415][INFRA][TEST] Regenerate benchmark results
29 июл 2026, 10:51
29 июл 2026, 10:51
6b793ce
Код
Авторство
О чём код?
================================================================================================ SQL Single Numeric Column Scan ================================================================================================ OpenJDK 64-Bit Server VM 17.0.20+8-LTS on Linux 6.17.0-1020-azure AMD EPYC 7763 64-Core Processor SQL Single BOOLEAN Column Scan: Best Time(ms) Avg Time(ms) Stdev(ms) Rate(M/s) Per Row(ns) Relative ------------------------------------------------------------------------------------------------------------------------ SQL CSV 12386 12408 32 1.3 787.5 1.0X SQL Json 8847 8971 176 1.8 562.5 1.4X SQL Json with UnsafeRow 9621 9720 140 1.6 611.7 1.3X SQL Parquet Vectorized: DataPageV1 104 117 10 150.7 6.6 118.7X SQL Parquet Vectorized: DataPageV2 107 116 7 147.6 6.8 116.2X SQL Parquet MR: DataPageV1 1781 1796 21 8.8 113.2 7.0X SQL Parquet MR: DataPageV2 1724 1725 1 9.1 109.6 7.2X SQL ORC Vectorized 147 153 5 106.8 9.4 84.1X SQL ORC MR 1560 1562 3 10.1 99.2 7.9X OpenJDK 64-Bit Server VM 17.0.20+8-LTS on Linux 6.17.0-1020-azure AMD EPYC 7763 64-Core Processor Parquet Reader Single BOOLEAN Column Scan: Best Time(ms) Avg Time(ms) Stdev(ms) Rate(M/s) Per Row(ns) Relative --------------------------------------------------------------------------------------------------------------------------- ParquetReader Vectorized: DataPageV1 89 90 1 176.2 5.7 1.0X ParquetReader Vectorized: DataPageV2 103 104 1 152.9 6.5 0.9X ParquetReader Vectorized -> Row: DataPageV1 67 68 1 235.7 4.2 1.3X ParquetReader Vectorized -> Row: DataPageV2 83 83 1 190.5 5.2 1.1X OpenJDK 64-Bit Server VM 17.0.20+8-LTS on Linux 6.17.0-1020-azure AMD EPYC 7763 64-Core Processor SQL Single TINYINT Column Scan: Best Time(ms) Avg Time(ms) Stdev(ms) Rate(M/s) Per Row(ns) Relative ------------------------------------------------------------------------------------------------------------------------ SQL CSV 12003 12017 19 1.3 763.1 1.0X SQL Json 9900 9908 11 1.6 629.4 1.2X SQL Json with UnsafeRow 10677 10686 13 1.5 678.8 1.1X SQL Parquet Vectorized: DataPageV1 138 142 4 114.3 8.7 87.2X SQL Parquet Vectorized: DataPageV2 134 140 4 117.7 8.5 89.8X SQL Parquet MR: DataPageV1 2051 2063 17 7.7 130.4 5.9X SQL Parquet MR: DataPageV2 1860 1871 17 8.5 118.2 6.5X SQL ORC Vectorized 105 111 6 149.4 6.7 114.0X SQL ORC MR 1496 1497 1 10.5 95.1 8.0X OpenJDK 64-Bit Server VM 17.0.20+8-LTS on Linux 6.17.0-1020-azure AMD EPYC 7763 64-Core Processor Parquet Reader Single TINYINT Column Scan: Best Time(ms) Avg Time(ms) Stdev(ms) Rate(M/s) Per Row(ns) Relative --------------------------------------------------------------------------------------------------------------------------- ParquetReader Vectorized: DataPageV1 146 146 0 107.7 9.3 1.0X ParquetReader Vectorized: DataPageV2 145 147 3 108.1 9.2 1.0X ParquetReader Vectorized -> Row: DataPageV1 48 49 1 330.7 3.0 3.1X ParquetReader Vectorized -> Row: DataPageV2 47 49 1 333.1 3.0 3.1X OpenJDK 64-Bit Server VM 17.0.20+8-LTS on Linux 6.17.0-1020-azure AMD EPYC 7763 64-Core Processor SQL Single SMALLINT Column Scan: Best Time(ms) Avg Time(ms) Stdev(ms) Rate(M/s) Per Row(ns) Relative ------------------------------------------------------------------------------------------------------------------------ SQL CSV 12516 12536 28 1.3 795.8 1.0X SQL Json 10038 10048 14 1.6 638.2 1.2X SQL Json with UnsafeRow 10803 10806 4 1.5 686.8 1.2X SQL Parquet Vectorized: DataPageV1 135 141 7 116.5 8.6 92.7X SQL Parquet Vectorized: DataPageV2 155 159 3 101.5 9.9 80.8X SQL Parquet MR: DataPageV1 2180 2193 19 7.2 138.6 5.7X SQL Parquet MR: DataPageV2 2000 2001 1 7.9 127.2 6.3X SQL ORC Vectorized 137 142 5 115.1 8.7 91.6X SQL ORC MR 1604 1604 0 9.8 102.0 7.8X OpenJDK 64-Bit Server VM 17.0.20+8-LTS on Linux 6.17.0-1020-azure AMD EPYC 7763 64-Core Processor Parquet Reader Single SMALLINT Column Scan: Best Time(ms) Avg Time(ms) Stdev(ms) Rate(M/s) Per Row(ns) Relative --------------------------------------------------------------------------------------------------------------------------- ParquetReader Vectorized: DataPageV1 165 168 5 95.5 10.5 1.0X ParquetReader Vectorized: DataPageV2 187 189 2 84.3 11.9 0.9X ParquetReader Vectorized -> Row: DataPageV1 162 164 3 97.3 10.3 1.0X ParquetReader Vectorized -> Row: DataPageV2 187 188 1 84.2 11.9 0.9X OpenJDK 64-Bit Server VM 17.0.20+8-LTS on Linux 6.17.0-1020-azure AMD EPYC 7763 64-Core Processor SQL Single INT Column Scan: Best Time(ms) Avg Time(ms) Stdev(ms) Rate(M/s) Per Row(ns) Relative ------------------------------------------------------------------------------------------------------------------------ SQL CSV 13751 14186 615 1.1 874.3 1.0X SQL Json 10595 10601 8 1.5 673.6 1.3X SQL Json with UnsafeRow 11148 11158 14 1.4 708.8 1.2X SQL Parquet Vectorized: DataPageV1 139 145 6 112.8 8.9 98.6X SQL Parquet Vectorized: DataPageV2 210 215 4 74.9 13.4 65.5X SQL Parquet MR: DataPageV1 2268 2271 4 6.9 144.2 6.1X SQL Parquet MR: DataPageV2 2198 2222 35 7.2 139.7 6.3X SQL ORC Vectorized 151 156 4 104.2 9.6 91.1X SQL ORC MR 1722 1728 9 9.1 109.5 8.0X OpenJDK 64-Bit Server VM 17.0.20+8-LTS on Linux 6.17.0-1020-azure AMD EPYC 7763 64-Core Processor Parquet Reader Single INT Column Scan: Best Time(ms) Avg Time(ms) Stdev(ms) Rate(M/s) Per Row(ns) Relative --------------------------------------------------------------------------------------------------------------------------- ParquetReader Vectorized: DataPageV1 157 158 1 100.4 10.0 1.0X ParquetReader Vectorized: DataPageV2 227 229 3 69.3 14.4 0.7X ParquetReader Vectorized -> Row: DataPageV1 171 172 2 92.2 10.8 0.9X ParquetReader Vectorized -> Row: DataPageV2 243 246 4 64.7 15.5 0.6X OpenJDK 64-Bit Server VM 17.0.20+8-LTS on Linux 6.17.0-1020-azure AMD EPYC 7763 64-Core Processor SQL Single BIGINT Column Scan: Best Time(ms) Avg Time(ms) Stdev(ms) Rate(M/s) Per Row(ns) Relative ------------------------------------------------------------------------------------------------------------------------ SQL CSV 13668 13713 62 1.2 869.0 1.0X SQL Json 10315 10323 12 1.5 655.8 1.3X SQL Json with UnsafeRow 10882 10886 7 1.4 691.8 1.3X SQL Parquet Vectorized: DataPageV1 297 308 11 52.9 18.9 46.0X SQL Parquet Vectorized: DataPageV2 184 190 5 85.4 11.7 74.2X SQL Parquet MR: DataPageV1 2583 2612 40 6.1 164.2 5.3X SQL Parquet MR: DataPageV2 2137 2162 35 7.4 135.9 6.4X SQL ORC Vectorized 157 162 3 100.0 10.0 86.9X SQL ORC MR 1781 1794 18 8.8 113.2 7.7X OpenJDK 64-Bit Server VM 17.0.20+8-LTS on Linux 6.17.0-1020-azure AMD EPYC 7763 64-Core Processor Parquet Reader Single BIGINT Column Scan: Best Time(ms) Avg Time(ms) Stdev(ms) Rate(M/s) Per Row(ns) Relative --------------------------------------------------------------------------------------------------------------------------- ParquetReader Vectorized: DataPageV1 359 361 2 43.8 22.8 1.0X ParquetReader Vectorized: DataPageV2 240 242 2 65.5 15.3 1.5X ParquetReader Vectorized -> Row: DataPageV1 359 366 9 43.8 22.9 1.0X ParquetReader Vectorized -> Row: DataPageV2 239 251 10 65.7 15.2 1.5X OpenJDK 64-Bit Server VM 17.0.20+8-LTS on Linux 6.17.0-1020-azure AMD EPYC 7763 64-Core Processor SQL Single FLOAT Column Scan: Best Time(ms) Avg Time(ms) Stdev(ms) Rate(M/s) Per Row(ns) Relative ------------------------------------------------------------------------------------------------------------------------ SQL CSV 13960 13968 11 1.1 887.6 1.0X SQL Json 11791 11792 1 1.3 749.7 1.2X SQL Json with UnsafeRow 12637 12653 23 1.2 803.4 1.1X SQL Parquet Vectorized: DataPageV1 104 107 3 151.3 6.6 134.3X SQL Parquet Vectorized: DataPageV2 103 107 3 152.2 6.6 135.1X SQL Parquet MR: DataPageV1 2146 2170 35 7.3 136.4 6.5X SQL Parquet MR: DataPageV2 2048 2052 6 7.7 130.2 6.8X SQL ORC Vectorized 233 238 5 67.4 14.8 59.8X SQL ORC MR 1805 1810 7 8.7 114.8 7.7X OpenJDK 64-Bit Server VM 17.0.20+8-LTS on Linux 6.17.0-1020-azure AMD EPYC 7763 64-Core Processor Parquet Reader Single FLOAT Column Scan: Best Time(ms) Avg Time(ms) Stdev(ms) Rate(M/s) Per Row(ns) Relative --------------------------------------------------------------------------------------------------------------------------- ParquetReader Vectorized: DataPageV1 161 162 1 97.8 10.2 1.0X ParquetReader Vectorized: DataPageV2 161 163 2 97.7 10.2 1.0X ParquetReader Vectorized -> Row: DataPageV1 160 163 2 98.0 10.2 1.0X ParquetReader Vectorized -> Row: DataPageV2 162 164 1 97.0 10.3 1.0X OpenJDK 64-Bit Server VM 17.0.20+8-LTS on Linux 6.17.0-1020-azure AMD EPYC 7763 64-Core Processor SQL Single DOUBLE Column Scan: Best Time(ms) Avg Time(ms) Stdev(ms) Rate(M/s) Per Row(ns) Relative ------------------------------------------------------------------------------------------------------------------------ SQL CSV 14257 14259 3 1.1 906.4 1.0X SQL Json 12002 12004 3 1.3 763.1 1.2X SQL Json with UnsafeRow 12808 12811 4 1.2 814.3 1.1X SQL Parquet Vectorized: DataPageV1 295 298 3 53.3 18.8 48.3X SQL Parquet Vectorized: DataPageV2 292 297 5 53.9 18.5 48.9X SQL Parquet MR: DataPageV1 2590 2606 23 6.1 164.7 5.5X SQL Parquet MR: DataPageV2 2443 2445 2 6.4 155.3 5.8X SQL ORC Vectorized 447 454 15 35.2 28.4 31.9X SQL ORC MR 2121 2129 12 7.4 134.8 6.7X OpenJDK 64-Bit Server VM 17.0.20+8-LTS on Linux 6.17.0-1020-azure AMD EPYC 7763 64-Core Processor Parquet Reader Single DOUBLE Column Scan: Best Time(ms) Avg Time(ms) Stdev(ms) Rate(M/s) Per Row(ns) Relative --------------------------------------------------------------------------------------------------------------------------- ParquetReader Vectorized: DataPageV1 354 355 1 44.5 22.5 1.0X ParquetReader Vectorized: DataPageV2 351 353 1 44.8 22.3 1.0X ParquetReader Vectorized -> Row: DataPageV1 374 374 1 42.1 23.8 0.9X ParquetReader Vectorized -> Row: DataPageV2 369 369 1 42.7 23.4 1.0X ================================================================================================ SQL Single Numeric Column Scan in Struct ================================================================================================ OpenJDK 64-Bit Server VM 17.0.20+8-LTS on Linux 6.17.0-1020-azure AMD EPYC 7763 64-Core Processor SQL Single TINYINT Column Scan in Struct: Best Time(ms) Avg Time(ms) Stdev(ms) Rate(M/s) Per Row(ns) Relative ------------------------------------------------------------------------------------------------------------------------------------------- SQL ORC MR 2108 2124 23 7.5 134.0 1.0X SQL ORC Vectorized (Nested Column Disabled) 2132 2134 3 7.4 135.5 1.0X SQL ORC Vectorized (Nested Column Enabled) 137 141 6 114.9 8.7 15.4X SQL Parquet MR: DataPageV1 2508 2560 75 6.3 159.4 0.8X SQL Parquet Vectorized: DataPageV1 (Nested Column Disabled) 3075 3080 7 5.1 195.5 0.7X SQL Parquet Vectorized: DataPageV1 (Nested Column Enabled) 100 104 3 156.7 6.4 21.0X SQL Parquet MR: DataPageV2 2441 2442 2 6.4 155.2 0.9X SQL Parquet Vectorized: DataPageV2 (Nested Column Disabled) 3093 3093 1 5.1 196.6 0.7X SQL Parquet Vectorized: DataPageV2 (Nested Column Enabled) 101 105 3 155.6 6.4 20.9X OpenJDK 64-Bit Server VM 17.0.20+8-LTS on Linux 6.17.0-1020-azure AMD EPYC 7763 64-Core Processor SQL Single SMALLINT Column Scan in Struct: Best Time(ms) Avg Time(ms) Stdev(ms) Rate(M/s) Per Row(ns) Relative ------------------------------------------------------------------------------------------------------------------------------------------- SQL ORC MR 2134 2147 19 7.4 135.6 1.0X SQL ORC Vectorized (Nested Column Disabled) 2119 2125 9 7.4 134.7 1.0X SQL ORC Vectorized (Nested Column Enabled) 159 164 3 99.1 10.1 13.4X SQL Parquet MR: DataPageV1 2525 2531 8 6.2 160.5 0.8X SQL Parquet Vectorized: DataPageV1 (Nested Column Disabled) 2987 2992 7 5.3 189.9 0.7X SQL Parquet Vectorized: DataPageV1 (Nested Column Enabled) 144 152 8 109.0 9.2 14.8X SQL Parquet MR: DataPageV2 2471 2473 3 6.4 157.1 0.9X SQL Parquet Vectorized: DataPageV2 (Nested Column Disabled) 2809 2810 1 5.6 178.6 0.8X SQL Parquet Vectorized: DataPageV2 (Nested Column Enabled) 161 166 4 97.5 10.3 13.2X OpenJDK 64-Bit Server VM 17.0.20+8-LTS on Linux 6.17.0-1020-azure AMD EPYC 7763 64-Core Processor SQL Single INT Column Scan in Struct: Best Time(ms) Avg Time(ms) Stdev(ms) Rate(M/s) Per Row(ns) Relative ------------------------------------------------------------------------------------------------------------------------------------------- SQL ORC MR 2105 2115 13 7.5 133.9 1.0X SQL ORC Vectorized (Nested Column Disabled) 2140 2140 1 7.4 136.0 1.0X SQL ORC Vectorized (Nested Column Enabled) 281 286 4 56.0 17.8 7.5X SQL Parquet MR: DataPageV1 2569 2570 1 6.1 163.3 0.8X SQL Parquet Vectorized: DataPageV1 (Nested Column Disabled) 3102 3103 1 5.1 197.2 0.7X SQL Parquet Vectorized: DataPageV1 (Nested Column Enabled) 120 125 6 131.6 7.6 17.6X SQL Parquet MR: DataPageV2 2496 2496 1 6.3 158.7 0.8X SQL Parquet Vectorized: DataPageV2 (Nested Column Disabled) 2928 2938 14 5.4 186.2 0.7X SQL Parquet Vectorized: DataPageV2 (Nested Column Enabled) 181 185 4 86.8 11.5 11.6X OpenJDK 64-Bit Server VM 17.0.20+8-LTS on Linux 6.17.0-1020-azure AMD EPYC 7763 64-Core Processor SQL Single BIGINT Column Scan in Struct: Best Time(ms) Avg Time(ms) Stdev(ms) Rate(M/s) Per Row(ns) Relative ------------------------------------------------------------------------------------------------------------------------------------------- SQL ORC MR 2076 2086 15 7.6 132.0 1.0X SQL ORC Vectorized (Nested Column Disabled) 2077 2118 58 7.6 132.0 1.0X SQL ORC Vectorized (Nested Column Enabled) 279 283 4 56.3 17.8 7.4X SQL Parquet MR: DataPageV1 2779 2798 26 5.7 176.7 0.7X SQL Parquet Vectorized: DataPageV1 (Nested Column Disabled) 3502 3511 13 4.5 222.7 0.6X SQL Parquet Vectorized: DataPageV1 (Nested Column Enabled) 311 400 50 50.6 19.7 6.7X SQL Parquet MR: DataPageV2 2373 2400 38 6.6 150.9 0.9X SQL Parquet Vectorized: DataPageV2 (Nested Column Disabled) 3001 3016 22 5.2 190.8 0.7X SQL Parquet Vectorized: DataPageV2 (Nested Column Enabled) 189 191 3 83.3 12.0 11.0X OpenJDK 64-Bit Server VM 17.0.20+8-LTS on Linux 6.17.0-1020-azure AMD EPYC 7763 64-Core Processor SQL Single FLOAT Column Scan in Struct: Best Time(ms) Avg Time(ms) Stdev(ms) Rate(M/s) Per Row(ns) Relative ------------------------------------------------------------------------------------------------------------------------------------------- SQL ORC MR 2240 2262 31 7.0 142.4 1.0X SQL ORC Vectorized (Nested Column Disabled) 2241 2247 9 7.0 142.5 1.0X SQL ORC Vectorized (Nested Column Enabled) 342 352 9 45.9 21.8 6.5X SQL Parquet MR: DataPageV1 2372 2380 11 6.6 150.8 0.9X SQL Parquet Vectorized: DataPageV1 (Nested Column Disabled) 3040 3050 14 5.2 193.3 0.7X SQL Parquet Vectorized: DataPageV1 (Nested Column Enabled) 192 196 4 82.0 12.2 11.7X SQL Parquet MR: DataPageV2 2295 2337 58 6.9 145.9 1.0X SQL Parquet Vectorized: DataPageV2 (Nested Column Disabled) 2924 2933 12 5.4 185.9 0.8X SQL Parquet Vectorized: DataPageV2 (Nested Column Enabled) 188 194 6 83.8 11.9 11.9X OpenJDK 64-Bit Server VM 17.0.20+8-LTS on Linux 6.17.0-1020-azure AMD EPYC 7763 64-Core Processor SQL Single DOUBLE Column Scan in Struct: Best Time(ms) Avg Time(ms) Stdev(ms) Rate(M/s) Per Row(ns) Relative ------------------------------------------------------------------------------------------------------------------------------------------- SQL ORC MR 2391 2458 94 6.6 152.0 1.0X SQL ORC Vectorized (Nested Column Disabled) 2345 2369 34 6.7 149.1 1.0X SQL ORC Vectorized (Nested Column Enabled) 588 591 3 26.7 37.4 4.1X SQL Parquet MR: DataPageV1 2736 2746 14 5.7 173.9 0.9X SQL Parquet Vectorized: DataPageV1 (Nested Column Disabled) 3379 3404 35 4.7 214.8 0.7X SQL Parquet Vectorized: DataPageV1 (Nested Column Enabled) 420 425 4 37.4 26.7 5.7X SQL Parquet MR: DataPageV2 2707 2713 8 5.8 172.1 0.9X SQL Parquet Vectorized: DataPageV2 (Nested Column Disabled) 3290 3333 61 4.8 209.2 0.7X SQL Parquet Vectorized: DataPageV2 (Nested Column Enabled) 380 383 3 41.4 24.2 6.3X ================================================================================================ SQL Nested Column Scan ================================================================================================ OpenJDK 64-Bit Server VM 17.0.20+8-LTS on Linux 6.17.0-1020-azure AMD EPYC 7763 64-Core Processor SQL Nested Column Scan: Best Time(ms) Avg Time(ms) Stdev(ms) Rate(M/s) Per Row(ns) Relative ------------------------------------------------------------------------------------------------------------------------------------------- SQL ORC MR 13329 13550 166 0.1 12711.7 1.0X SQL ORC Vectorized (Nested Column Disabled) 13223 13407 159 0.1 12610.2 1.0X SQL ORC Vectorized (Nested Column Enabled) 7383 7420 23 0.1 7040.9 1.8X SQL Parquet MR: DataPageV1 8658 8697 23 0.1 8257.3 1.5X SQL Parquet Vectorized: DataPageV1 (Nested Column Disabled) 9034 9080 31 0.1 8615.2 1.5X SQL Parquet Vectorized: DataPageV1 (Nested Column Enabled) 5725 5758 30 0.2 5459.4 2.3X SQL Parquet MR: DataPageV2 9549 9571 12 0.1 9107.0 1.4X SQL Parquet Vectorized: DataPageV2 (Nested Column Disabled) 9868 9901 30 0.1 9411.0 1.4X SQL Parquet Vectorized: DataPageV2 (Nested Column Enabled) 5486 5577 78 0.2 5231.8 2.4X ================================================================================================ Int and String Scan ================================================================================================ OpenJDK 64-Bit Server VM 17.0.20+8-LTS on Linux 6.17.0-1020-azure AMD EPYC 7763 64-Core Processor Int and String Scan: Best Time(ms) Avg Time(ms) Stdev(ms) Rate(M/s) Per Row(ns) Relative ------------------------------------------------------------------------------------------------------------------------ SQL CSV 12279 12460 256 0.9 1171.1 1.0X SQL Json 10547 10568 30 1.0 1005.8 1.2X SQL Parquet Vectorized: DataPageV1 1770 1772 3 5.9 168.8 6.9X SQL Parquet Vectorized: DataPageV2 1851 1870 27 5.7 176.5 6.6X SQL Parquet MR: DataPageV1 4042 4057 22 2.6 385.5 3.0X SQL Parquet MR: DataPageV2 3871 3878 10 2.7 369.1 3.2X SQL ORC Vectorized 1569 1605 51 6.7 149.6 7.8X SQL ORC MR 3259 3268 12 3.2 310.8 3.8X ================================================================================================ Repeated String Scan ================================================================================================ OpenJDK 64-Bit Server VM 17.0.20+8-LTS on Linux 6.17.0-1020-azure AMD EPYC 7763 64-Core Processor Repeated String: Best Time(ms) Avg Time(ms) Stdev(ms) Rate(M/s) Per Row(ns) Relative ------------------------------------------------------------------------------------------------------------------------ SQL CSV 7319 7327 12 1.4 698.0 1.0X SQL Json 6964 6975 16 1.5 664.1 1.1X SQL Parquet Vectorized: DataPageV1 475 483 12 22.1 45.3 15.4X SQL Parquet Vectorized: DataPageV2 477 481 3 22.0 45.5 15.3X SQL Parquet MR: DataPageV1 1712 1717 6 6.1 163.3 4.3X SQL Parquet MR: DataPageV2 1657 1670 19 6.3 158.0 4.4X SQL ORC Vectorized 381 385 3 27.5 36.3 19.2X SQL ORC MR 1823 1837 19 5.8 173.9 4.0X ================================================================================================ Partitioned Table Scan ================================================================================================ OpenJDK 64-Bit Server VM 17.0.20+8-LTS on Linux 6.17.0-1020-azure AMD EPYC 7763 64-Core Processor Partitioned Table: Best Time(ms) Avg Time(ms) Stdev(ms) Rate(M/s) Per Row(ns) Relative --------------------------------------------------------------------------------------------------------------------------------- Data column - CSV 13631 13725 133 1.2 866.6 1.0X Data column - Json 10162 10172 14 1.5 646.1 1.3X Data column - Parquet Vectorized: DataPageV1 114 119 3 137.5 7.3 119.2X Data column - Parquet Vectorized: DataPageV2 198 202 3 79.4 12.6 68.8X Data column - Parquet MR: DataPageV1 2532 2563 44 6.2 161.0 5.4X Data column - Parquet MR: DataPageV2 2259 2262 4 7.0 143.6 6.0X Data column - ORC Vectorized 155 157 2 101.8 9.8 88.2X Data column - ORC MR 1969 1970 2 8.0 125.2 6.9X Partition column - CSV 3504 3510 8 4.5 222.8 3.9X Partition column - Json 8920 8924 5 1.8 567.1 1.5X Partition column - Parquet Vectorized: DataPageV1 31 34 4 512.0 2.0 443.7X Partition column - Parquet Vectorized: DataPageV2 30 33 3 517.1 1.9 448.2X Partition column - Parquet MR: DataPageV1 1279 1280 2 12.3 81.3 10.7X Partition column - Parquet MR: DataPageV2 1267 1267 1 12.4 80.5 10.8X Partition column - ORC Vectorized 31 34 4 500.6 2.0 433.8X Partition column - ORC MR 1225 1253 39 12.8 77.9 11.1X Both columns - CSV 13573 13598 35 1.2 862.9 1.0X Both columns - Json 10566 10611 64 1.5 671.7 1.3X Both columns - Parquet Vectorized: DataPageV1 110 117 5 143.0 7.0 123.9X Both columns - Parquet Vectorized: DataPageV2 171 176 3 91.9 10.9 79.7X Both columns - Parquet MR: DataPageV1 2334 2349 22 6.7 148.4 5.8X Both columns - Parquet MR: DataPageV2 2281 2290 13 6.9 145.0 6.0X Both columns - ORC Vectorized 159 167 7 98.7 10.1 85.5X Both columns - ORC MR 1931 1934 4 8.1 122.8 7.1X ================================================================================================ String with Nulls Scan ================================================================================================ OpenJDK 64-Bit Server VM 17.0.20+8-LTS on Linux 6.17.0-1020-azure AMD EPYC 7763 64-Core Processor String with Nulls Scan (0.0%): Best Time(ms) Avg Time(ms) Stdev(ms) Rate(M/s) Per Row(ns) Relative ------------------------------------------------------------------------------------------------------------------------ SQL CSV 8866 8892 37 1.2 845.6 1.0X SQL Json 9271 9286 21 1.1 884.1 1.0X SQL Parquet Vectorized: DataPageV1 1264 1264 1 8.3 120.5 7.0X SQL Parquet Vectorized: DataPageV2 1313 1314 1 8.0 125.2 6.8X SQL Parquet MR: DataPageV1 3356 3358 3 3.1 320.0 2.6X SQL Parquet MR: DataPageV2 3407 3412 7 3.1 324.9 2.6X ParquetReader Vectorized: DataPageV1 868 870 2 12.1 82.8 10.2X ParquetReader Vectorized: DataPageV2 921 923 2 11.4 87.9 9.6X SQL ORC Vectorized 780 787 6 13.4 74.4 11.4X SQL ORC MR 2720 2740 27 3.9 259.4 3.3X OpenJDK 64-Bit Server VM 17.0.20+8-LTS on Linux 6.17.0-1020-azure AMD EPYC 7763 64-Core Processor String with Nulls Scan (50.0%): Best Time(ms) Avg Time(ms) Stdev(ms) Rate(M/s) Per Row(ns) Relative ------------------------------------------------------------------------------------------------------------------------ SQL CSV 6351 6363 16 1.7 605.7 1.0X SQL Json 7849 7850 1 1.3 748.5 0.8X SQL Parquet Vectorized: DataPageV1 865 869 5 12.1 82.5 7.3X SQL Parquet Vectorized: DataPageV2 930 935 7 11.3 88.7 6.8X SQL Parquet MR: DataPageV1 2911 2912 1 3.6 277.7 2.2X SQL Parquet MR: DataPageV2 2908 2914 9 3.6 277.4 2.2X ParquetReader Vectorized: DataPageV1 757 760 5 13.8 72.2 8.4X ParquetReader Vectorized: DataPageV2 817 818 1 12.8 77.9 7.8X SQL ORC Vectorized 984 994 10 10.7 93.8 6.5X SQL ORC MR 2778 2780 4 3.8 264.9 2.3X OpenJDK 64-Bit Server VM 17.0.20+8-LTS on Linux 6.17.0-1020-azure AMD EPYC 7763 64-Core Processor String with Nulls Scan (95.0%): Best Time(ms) Avg Time(ms) Stdev(ms) Rate(M/s) Per Row(ns) Relative ------------------------------------------------------------------------------------------------------------------------ SQL CSV 4372 4385 18 2.4 416.9 1.0X SQL Json 5629 5638 13 1.9 536.9 0.8X SQL Parquet Vectorized: DataPageV1 178 182 5 59.0 16.9 24.6X SQL Parquet Vectorized: DataPageV2 191 195 4 55.0 18.2 22.9X SQL Parquet MR: DataPageV1 1877 1879 2 5.6 179.0 2.3X SQL Parquet MR: DataPageV2 1783 1787 6 5.9 170.0 2.5X ParquetReader Vectorized: DataPageV1 184 186 1 56.9 17.6 23.7X ParquetReader Vectorized: DataPageV2 199 200 1 52.7 19.0 22.0X SQL ORC Vectorized 314 319 6 33.4 30.0 13.9X SQL ORC MR 1648 1650 3 6.4 157.1 2.7X ================================================================================================ Single Column Scan From Wide Columns ================================================================================================ OpenJDK 64-Bit Server VM 17.0.20+8-LTS on Linux 6.17.0-1020-azure AMD EPYC 7763 64-Core Processor Single Column Scan from 10 columns: Best Time(ms) Avg Time(ms) Stdev(ms) Rate(M/s) Per Row(ns) Relative ------------------------------------------------------------------------------------------------------------------------ SQL CSV 1194 1198 7 0.9 1138.4 1.0X SQL Json 1735 1737 2 0.6 1654.9 0.7X SQL Parquet Vectorized: DataPageV1 27 31 4 39.2 25.5 44.6X SQL Parquet Vectorized: DataPageV2 31 34 3 34.0 29.4 38.8X SQL Parquet MR: DataPageV1 175 181 3 6.0 167.2 6.8X SQL Parquet MR: DataPageV2 173 177 4 6.1 165.0 6.9X SQL ORC Vectorized 30 34 4 34.9 28.7 39.7X SQL ORC MR 127 129 3 8.2 121.2 9.4X OpenJDK 64-Bit Server VM 17.0.20+8-LTS on Linux 6.17.0-1020-azure AMD EPYC 7763 64-Core Processor Single Column Scan from 50 columns: Best Time(ms) Avg Time(ms) Stdev(ms) Rate(M/s) Per Row(ns) Relative ------------------------------------------------------------------------------------------------------------------------ SQL CSV 2622 2626 5 0.4 2501.0 1.0X SQL Json 6042 6130 125 0.2 5761.7 0.4X SQL Parquet Vectorized: DataPageV1 29 33 5 36.4 27.5 91.1X SQL Parquet Vectorized: DataPageV2 33 35 4 32.2 31.0 80.6X SQL Parquet MR: DataPageV1 174 178 3 6.0 166.0 15.1X SQL Parquet MR: DataPageV2 187 192 2 5.6 178.8 14.0X SQL ORC Vectorized 33 35 4 32.0 31.2 80.1X SQL ORC MR 131 134 3 8.0 124.6 20.1X OpenJDK 64-Bit Server VM 17.0.20+8-LTS on Linux 6.17.0-1020-azure AMD EPYC 7763 64-Core Processor Single Column Scan from 100 columns: Best Time(ms) Avg Time(ms) Stdev(ms) Rate(M/s) Per Row(ns) Relative ------------------------------------------------------------------------------------------------------------------------ SQL CSV 4375 4379 7 0.2 4172.0 1.0X SQL Json 11602 11766 232 0.1 11064.6 0.4X SQL Parquet Vectorized: DataPageV1 38 42 5 27.7 36.1 115.6X SQL Parquet Vectorized: DataPageV2 42 46 5 24.9 40.2 103.9X SQL Parquet MR: DataPageV1 189 194 5 5.5 180.3 23.1X SQL Parquet MR: DataPageV2 201 204 3 5.2 191.7 21.8X SQL ORC Vectorized 42 45 4 25.3 39.6 105.4X SQL ORC MR 142 145 4 7.4 135.1 30.9X