storm-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From clay teahouse <clayteaho...@gmail.com>
Subject Re: the external Zookeeper in Local mode
Date Wed, 04 Feb 2015 10:54:31 GMT
There was a mistake in my code and now the topology is using the external
zookeeper. But I still see the message
No partition information found, using configuration to determine offset

I don't know the significance of this message. My main objective is to find
out why kafka spout is so slow.

thanks
Clay

On Wed, Feb 4, 2015 at 4:04 AM, clay teahouse <clayteahouse@gmail.com>
wrote:

> Hi,
> Even if I use the local cluster with the constructor that specified the
> external zookeeper's address, I still see:
> backtype.storm.zookeeper - Starting inprocess zookeeper at port 2000
> and
>  org.apache.storm.zookeeper.ZooKeeper - Initiating client connection,
> connectString=localhost:2000 sessionTimeout=20000
> watcher=org.apache.storm.curator.ConnectionState@7887fb2f
>
> And I still see the message:
>
> No partition information found, using configuration to determine offset
>
> I am mostly concerned about the latter message. Why am I getting this
> message?
>
> thanks,
> Clay
>
>
> On Tue, Feb 3, 2015 at 11:12 PM, Harsha <storm@harsha.io> wrote:
>
>> LocalCluster should be used for debugging a topology . There is another
>> constructor you can use
>> LocalCluster cluster = new LocalCluster("localhost", new Long(2182));
>> first param is zookeeper host and second is the port.
>> -Harsha
>>
>> On Tue, Feb 3, 2015, at 07:48 PM, Shivendra Singh wrote:
>> > Hi Clay,
>> >
>> > The zookeeper info provided to KafkaSpout is external zookeeper which is
>> > used by Kafka and where all the host/partition information for Kafka is
>> > stored.
>> >
>> > When you run storm in local mode, it spins up an in-memory zookeeper for
>> > storm where it keeps data related to nimbus/supervisors, etc.
>> >
>> > Thanks,
>> > Shivendra
>> >
>> >
>> > > On Feb 3, 2015, at 7:23 PM, clay teahouse <clayteahouse@gmail.com>
>> wrote:
>> > >
>> > > Hi,
>> > >
>> > > I have a topology running in local mode. The topology uses kafkaspout
>> and is configured to use the external zookeeper. But when I start the
>> topology, I see the following:
>> > >
>> > >  org.apache.storm.zookeeper.ZooKeeper - Initiating client connection,
>> connectString=localhost:2000
>> > >
>> > > I also messages like the following:
>> > >
>> > > No partition information found, using configuration to determine
>> offset
>> > >
>> > > I am not sure if these two messages are related.
>> > >
>> > > Questions:
>> > > 1) Why storm is employing the in-memory zookeeper when I have
>> configured the spout to use the external zookeeper.
>> > > 2)How do I force storm to use the external zookeeper?
>> > > 3) Why am I getting the message No partition information found, using
>> configuration to determine offset? What configuration is it referring to?
>> > >
>> > > thanks,
>> > > Clay
>> > >
>>
>
>

Mime
View raw message