spark-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Michael Mior <mm...@uwaterloo.ca>
Subject Task partition ID in Spark event logs
Date Thu, 20 Jul 2017 15:44:14 GMT
I see there's a comment in the TaskInfo class that the index may not be the
same as the ID of the RDD partition the task is computing. Under what
circumstances *will* the ID by the same? If there are zero guarantees, any
suggestions on how to grab this info from the scheduler to populate a new
field inside TaskInfo?

Cheers,
--
Michael Mior
mmior@apache.org

Mime
View raw message