spark-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Priyanka Gomatam <>
Subject Spark-Locality: Hinting Spark location of the executor does not take effect
Date Wed, 16 Sep 2020 23:34:31 GMT
Sending on behalf of a colleague whose mail isn’t reaching the dev list for some reason 😊


HI Spark developers,

If I want to hint spark to use particular list of hosts to execute tasks on. I see that getBlockLocations
is used to get the list of hosts from HDFS.<>

Hinting Spark by custom getBlockLocation which return Array of BlockLocations with host ip
address doesn’t help, Spark continues to host it on other executors hosts.

Is there something I am doing wrong ?


Appreciate your inputs 😊


View raw message