spark-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From nakhanha <nakha...@microsoft.com.INVALID>
Subject Re: Spark-Locality: Hinting Spark location of the executor does not take effect.
Date Thu, 17 Sep 2020 01:40:26 GMT
Even though i provide hint of wn4's ip address

Spark schedules on wn1
2020-09-17 01:21:50,038 DEBUG [dag-scheduler-event-loop]
scheduler.TaskSetManager: Valid locality levels for TaskSet 11.0:
NODE_LOCAL, RACK_LOCAL, ANY
2020-09-17 01:21:50,039 DEBUG [dispatcher-event-loop-0]
cluster.YarnScheduler: parentName: , name: TaskSet_11.0, runningTasks: 0
2020-09-17 01:21:50,040 INFO  [dispatcher-event-loop-0]
scheduler.TaskSetManager: Starting task 0.0 in stage 11.0 (TID 13,
wn1-vegasr.r0erhw3gxezevknl0vbc42vctb.dx.internal.cloudapp.net, executor 6,
partition 5, NODE_LOCAL, 4994 bytes)
2020-09-17 01:21:50,040 DEBUG [dispatcher-event-loop-0]
scheduler.TaskSetManager: No tasks for locality level NODE_LOCAL, so moving
to locality level RACK_LOCAL
2020-09-17 01:21:50,041 DEBUG [dispatcher-event-loop-0]
scheduler.TaskSetManager: No tasks for locality level RACK_LOCAL, so moving
to locality level ANY
2020-09-17 01:21:50,042 DEBUG [dispatcher-event-loop-0]
cluster.YarnSchedulerBackend$YarnDriverEndpoint: Launching task 13 on
executor id: 6 hostname:
wn1-vegasr.r0erhw3gxezevknl0vbc42vctb.dx.internal.cloudapp.net.





--
Sent from: http://apache-spark-developers-list.1001551.n3.nabble.com/

---------------------------------------------------------------------
To unsubscribe e-mail: dev-unsubscribe@spark.apache.org


Mime
View raw message