unify benchmarks in 2.4 and regenerate results

classic Classic list List threaded Threaded
1 message Options
Reply | Threaded
Open this post in threaded view
|

unify benchmarks in 2.4 and regenerate results

Maxim Gekk
Hi All,

Currently, most of benchmark results are embedded into benchmark source codes in Spark 2.4.x. This makes comparison of the results between 2.4 releases and master pretty inconvenient. I would like to propose to unify benchmarks in branch-2.4 by backporting the changes made in the master: https://issues.apache.org/jira/browse/SPARK-25475 and regenerate all results in the same environment. This will allow to compare Spark 3.0 to Spark 2.4.x, and minor releases of 2.4.x.

Maxim Gekk

Software Engineer

Databricks, Inc.