spark-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From hemant singh <hemant2...@gmail.com>
Subject Re: Spark onApplicationEnd run multiple times during the application failure
Date Thu, 21 Nov 2019 08:12:02 GMT
Could it be because of re-try.

Thanks

On Thu, 21 Nov 2019 at 3:35 AM, Jiang, Yi J (CWM-NR)
<yi.j.jiang@rbc.com.invalid> wrote:

> Hello
>
> We are running into an issue.
>
> We have customized the SparkListener class, and added that to spark
> context. But when the spark job is failed, we find the “onApplicationEnd”
> function was triggered twice.
>
> Is that supposed to be triggered just once when the spark job is failed?
> Because the application level of spark job is only launched once, how can
> it be triggered twice when it is failed.
>
> Please let us know
>
> Thank you
>
>
>
>
>
> _______________________________________________________________________
>
> This email is intended only for the use of the individual(s) to whom it is
> addressed and may be privileged and confidential.
> Unauthorised use or disclosure is prohibited. If you receive this e-mail
> in error, please advise immediately and delete the original message.
> This message may have been altered without your or our knowledge and the
> sender does not accept any liability for any errors or omissions in the
> message.
>
> Ce courriel est confidentiel et protégé. L'expéditeur ne renonce pas aux
> droits et obligations qui s'y rapportent.
> Toute diffusion, utilisation ou copie de ce message ou des renseignements
> qu'il contient par une personne autre que le (les) destinataire(s)
> désigné(s) est interdite.
> Si vous recevez ce courriel par erreur, veuillez m'en aviser
> immédiatement, par retour de courriel ou par un autre moyen.
>

Mime
View raw message