spark-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From hemant singh <>
Subject Re: Spark onApplicationEnd run multiple times during the application failure
Date Thu, 21 Nov 2019 08:12:02 GMT
Could it be because of re-try.


On Thu, 21 Nov 2019 at 3:35 AM, Jiang, Yi J (CWM-NR)
<> wrote:

> Hello
> We are running into an issue.
> We have customized the SparkListener class, and added that to spark
> context. But when the spark job is failed, we find the “onApplicationEnd”
> function was triggered twice.
> Is that supposed to be triggered just once when the spark job is failed?
> Because the application level of spark job is only launched once, how can
> it be triggered twice when it is failed.
> Please let us know
> Thank you
> _______________________________________________________________________
> This email is intended only for the use of the individual(s) to whom it is
> addressed and may be privileged and confidential.
> Unauthorised use or disclosure is prohibited. If you receive this e-mail
> in error, please advise immediately and delete the original message.
> This message may have been altered without your or our knowledge and the
> sender does not accept any liability for any errors or omissions in the
> message.
> Ce courriel est confidentiel et protégé. L'expéditeur ne renonce pas aux
> droits et obligations qui s'y rapportent.
> Toute diffusion, utilisation ou copie de ce message ou des renseignements
> qu'il contient par une personne autre que le (les) destinataire(s)
> désigné(s) est interdite.
> Si vous recevez ce courriel par erreur, veuillez m'en aviser
> immédiatement, par retour de courriel ou par un autre moyen.

View raw message