storm-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From pradeep s <sreekumar.prad...@gmail.com>
Subject Re: Storm Kafka offset monitoring
Date Wed, 22 Feb 2017 01:14:37 GMT
Hi Priyank
Currently I tested the spout using zookeeper broker hosts .This is
processing fine.But I saw another way of initialising spout using Kafka
bootstrap severs using kafkaspoutconfig class.


https://storm.apache.org/releases/1.0.2/javadocs/org/apache/storm/kafka/spout/KafkaSpoutConfig.Builder.html



But implementing this way I was getting commit failed exception due to
rebalance
Can you point out what's the proper way for implementing Kafka spout.
In storm 1.0.3 docs I have seen the way using zookeeper broker hosts
Regards
Pradeep S
On Tue, Feb 21, 2017 at 2:35 PM Priyank Shah <pshah@hortonworks.com> wrote:

> Hi Pradeep,
>
>
>
> A release vote for RC1 of 1.1.0 is in progress. You can track that and
> once it gets released you can upgrade.
>
>
>
> Regarding upgrading your spout, you don’t need to make any code changes.
> You just need to use the latest released spout code. Usually it involves
> updating a pom file that has a dependency on storm-kafka module to latest
> version.
>
>
>
> *From: *pradeep s <sreekumar.pradeep@gmail.com>
> *Reply-To: *"user@storm.apache.org" <user@storm.apache.org>
> *Date: *Tuesday, February 21, 2017 at 12:49 PM
> *To: *"user@storm.apache.org" <user@storm.apache.org>
> *Subject: *Re: Storm Kafka offset monitoring
>
>
>
> Hi Priyank
>
> Thanks for your reply.i was not able to find 1.1.0 version of storm
>
> Can you please point to that.also can you please  confirm on what specific
> spout changes to make.
>
> Regards
>
> Pradeep S
>
> On Tue, Feb 21, 2017 at 10:54 AM Priyank Shah <pshah@hortonworks.com>
> wrote:
>
> Hi Pradeep,
>
>
>
> If you upgrade your spout in the topology and storm code to a later
> version(I checked v1.1.0 and it has the tool) you will get a table in storm
> ui which show you offsets. If you cannot upgrade then I think you will have
> to do it manually.
>
>
>
> *From: *pradeep s <sreekumar.pradeep@gmail.com>
> *Reply-To: *"user@storm.apache.org" <user@storm.apache.org>
> *Date: *Tuesday, February 21, 2017 at 9:44 AM
> *To: *"user@storm.apache.org" <user@storm.apache.org>
> *Subject: *Storm Kafka offset monitoring
>
>
>
> Hi ,
>
> I am using Storm 1.0.2 and Kafka 0.10.1.1 versions . Storm spout is
> configured using zookeeper broker hosts. Is there a monitoring ui which i
> can use to track the consumer offsets and lag.
>
> I was using yahoo kafka manager , but its showing storm spout as a
> consumer.Any help?
>
> Regards
>
> Pradeep S
>
>

Mime
View raw message