spark-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Aniket More <>
Subject Identifying DataFrames in executing tasks
Date Wed, 07 Dec 2016 09:27:59 GMT
I am doing a POC in which I have implemented custom Spark Listener.
I have overridden methods such as onTaskEnd(taskEnd:
from which I get information such as
But I am not able to identify the dataframe executing in a task. 
For eg: I need to identify the dataframe where input file is read or task in
which dataframes are joined.

Can someone suggest me some solution for above use case where I can get
dataframes information in executing tasks?

View this message in context:
Sent from the Apache Spark User List mailing list archive at

To unsubscribe e-mail:

View raw message