spark-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Tathagata Das (JIRA)" <>
Subject [jira] [Commented] (SPARK-4802) ReceiverInfo removal at ReceiverTracker upon deregistering receiver
Date Tue, 23 Dec 2014 23:13:13 GMT


Tathagata Das commented on SPARK-4802:

SPARK-2892 is not a duplicate of this though they have same symptoms, the reason are different
but related. SPARK-2892 affects only socket receiver and is probably because the socket receiver
does not stop cleanly. SPARK-4802 affects all receivers, and prevents the driver from realizing
that all receivers have been closed.

> ReceiverInfo removal at ReceiverTracker upon deregistering receiver
> -------------------------------------------------------------------
>                 Key: SPARK-4802
>                 URL:
>             Project: Spark
>          Issue Type: Bug
>          Components: Streaming
>    Affects Versions: 1.1.0
>            Reporter: Ilayaperumal Gopinathan
>            Priority: Minor
> When the streaming receiver is deregistered, the ReceiverTracker doesn't remove the corresponding
receiverInfo entry for the receiver.
> When the receiver is stopped at the executor and the ReceiverTrackerActor that processes
the 'DeregisterReceiver' message. Shouldn't it remove the receiverInfo entry for that receiver
as the receiver is actually deregistered?
> Not sure if there is any specific reason for not removing it.
> Currently, I see this warning if we don't remove it:
> WARN main-EventThread scheduler.ReceiverTracker - All of the receivers have not deregistered,
Map(0 -> ReceiverInfo(0,MyReceiver-0,null,false,localhost,Stopped by driver,))

This message was sent by Atlassian JIRA

To unsubscribe, e-mail:
For additional commands, e-mail:

View raw message