storm-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Abhishek Agarwal <abhishc...@gmail.com>
Subject Re: Topology gets stuck
Date Thu, 09 Jun 2016 13:55:40 GMT
If spout is blocked only for a short duration, it may be due to slow bolt.
If it remains blocked forever, there is a possibility of deadlock. you can
check out STORM-1027

On Thu, Jun 9, 2016 at 6:48 PM, Nitin Gupta <Nitin.Gupta@e-zest.in> wrote:

> Hi Abhishek,
>
>
>
> I am using version 0.9.4 .
>
>
>
> From the log it seems the topology keeps processing a fix set of messages
> from the last offset it got stuck . Looks it is equal to the max spout
> pending messages. It processes these messages but doesn’t update the
> zookeeper with the processed offset. It then stop for a few minutes and
> repeats the same process.
>
>
>
> The rate at which the messages are being written is very high . So the lag
> is very high . Not sure if the KafkaSpout stops processing after a specific
> number of messages pending to be processed.
>
>
>
> Thanks & Regards,
>
> Nitin Gupta
>
>
>
> *From:* Abhishek Agarwal [mailto:abhishcool@gmail.com]
> *Sent:* Thursday, June 09, 2016 6:40 PM
> *To:* user@storm.apache.org
> *Subject:* Re: Topology gets stuck
>
>
>
> check the thread dump of your worker process after it gets stuck. which
> version do you use?
>
>
>
> On Thu, Jun 9, 2016 at 6:36 PM, Nitin Gupta <Nitin.Gupta@e-zest.in> wrote:
>
> Dear All,
>
>
>
> I am using a Kafkaspout to process messages from Kafka. The bolt takes
> around 700 milliseconds to process the message.
>
> I observe the topology worked fine for a few hours but once the number of
> pending messages increases it stops to update the consumer offset in the
> zookeeper.
>
>
>
> I am facing this problem from quite few days. Any help will be highly
> appreciated.
>
>
>
> Thanks & Regards,
>
> Nitin Gupta
>
>
>
>
>
>
>
> --
>
> Regards,
> Abhishek Agarwal
>
>
>



-- 
Regards,
Abhishek Agarwal

Mime
View raw message