Research Article
Query Execution Optimization in Spark SQL
Table 1
Comparison between calculated and actual shuffle sizes.
| ID | 1 | 2 | 3 | 4 | 5 | 6 | 7 | 8 | 9 | 10 | 11 |
| General task | 9 | 114 | 14 | 119 | 7 | 14 | 9 | 22 | 14 | 7 | 7 | Input (MB) | 256 | 2688 | 384 | 2560 | 512 | 384 | 128 | 128 | 256 | 640 | 256 | Calculated shuffle size | 27 | 282.4 | 40.3 | 269 | 53.81 | 40.35 | 13.45 | 15.05 | 28.03 | 67.25 | 29.06 | Actual shuffle size | 25 | 267.58 | 38.2 | 254.8 | 50.96 | 38.22 | 12.76 | 12.76 | 25.48 | 63.71 | 25.48 |
|
|