spark-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Jiang, Yi J (CWM-NR)" <>
Subject Spark onApplicationEnd run multiple times during the application failure
Date Wed, 20 Nov 2019 22:05:15 GMT
We are running into an issue.
We have customized the SparkListener class, and added that to spark context. But when the
spark job is failed, we find the "onApplicationEnd" function was triggered twice.
Is that supposed to be triggered just once when the spark job is failed? Because the application
level of spark job is only launched once, how can it be triggered twice when it is failed.
Please let us know
Thank you


This email is intended only for the use of the individual(s) to whom it is addressed and may
be privileged and confidential.
Unauthorised use or disclosure is prohibited. If you receive this e-mail in error, please
advise immediately and delete the original message. 
This message may have been altered without your or our knowledge and the sender does not accept
any liability for any errors or omissions in the message.

Ce courriel est confidentiel et protégé. L'expéditeur ne renonce pas aux droits et obligations
qui s'y rapportent. 
Toute diffusion, utilisation ou copie de ce message ou des renseignements qu'il contient par
une personne autre que le (les) destinataire(s) désigné(s) est interdite.
Si vous recevez ce courriel par erreur, veuillez m'en aviser immédiatement, par retour de
courriel ou par un autre moyen.
View raw message