spark-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Michael Mior <>
Subject Task partition ID in Spark event logs
Date Thu, 20 Jul 2017 15:44:14 GMT
I see there's a comment in the TaskInfo class that the index may not be the
same as the ID of the RDD partition the task is computing. Under what
circumstances *will* the ID by the same? If there are zero guarantees, any
suggestions on how to grab this info from the scheduler to populate a new
field inside TaskInfo?

Michael Mior

View raw message