spark-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "ramannanda9@gmail.com" <ramannan...@gmail.com>
Subject Is dataframe write blocking? what can be done for fair scheduler?
Date Thu, 08 Nov 2018 15:18:41 GMT
Hi,

I have noticed that in fair scheduler setting, if i block on dataframe write
to complete, using AwaitResult, the API call ends up returning, whereas that
is not what I intend to do as it can cause inconsistencies later in the
pipeline. Is there a way to make the dataframe write call blocking?


Regards,
Ramandeep Singh



--
Sent from: http://apache-spark-user-list.1001560.n3.nabble.com/

---------------------------------------------------------------------
To unsubscribe e-mail: user-unsubscribe@spark.apache.org


Mime
View raw message