Sending on behalf of a colleague whose mail isn’t reaching the dev list for some reason
HI Spark developers,
If I want to hint spark to use particular list of hosts to execute tasks on. I see that getBlockLocations is used to get the list of hosts from HDFS.
Hinting Spark by custom getBlockLocation which return Array of BlockLocations with host ip address doesn’t help, Spark continues to host it on other executors hosts.
Is there something I am doing wrong ?
Appreciate your inputs