This is the bi-weekly Apache Spark digest from the Databricks OSS team (this time we combined 4 weeks).
For each API/configuration/behavior change, an [API] tag is added in the title.
All the content has been appended to this doc.
[API][SPARK-32283][CORE] Kryo should support multiple user registrators (+13, -12)>
[SPARK-32175][CORE] Fix the order between initialization for ExecutorPlugin and starting heartbeat thread (+89, -10)>
[API][SPARK-32160][CORE][PYSPARK] Add a config to switch allow/disallow to create SparkContext in executors (+72, -9)>
[SPARK-32199][SPARK-32198] Reduce job failures during decommissioning (+539, -7)>
[SPARK-31197][CORE] Shutdown executor once we are done decommissioning (+310, -44)>
[SPARK-32518][CORE] CoarseGrainedSchedulerBackend.maxNumConcurrentTasks should consider all kinds of resources (+133, -59)>
[API][SPARK-32517][CORE] Add StorageLevel.DISK_ONLY_3 (+17, -4)>
[SPARK-31198][CORE] Use graceful decommissioning as part of dynamic scaling (+380, -134)>
[SPARK-32119][CORE] ExecutorPlugin doesn't work with Standalone Cluster and Kubernetes with --jars (+122, -9)>
[API][SPARK-32613][CORE] Fix regressions in DecommissionWorkerSuite (+153, -37)>
[API][SPARK-32651][CORE] Decommission switch configuration should have the highest hierarchy (+30, -25)>
[SPARK-32600][CORE] Unify task name in some logs between driver and executor (+44, -42)>
|Free forum by Nabble||Edit this page|