chukwa-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Sourygna Luangsay (JIRA)" <>
Subject [jira] [Commented] (CHUKWA-670) Use ZooKeeper for collector list and adaptor check list
Date Thu, 29 Nov 2012 06:00:58 GMT


Sourygna Luangsay commented on CHUKWA-670:

Ari has a point.

I see 2 other drawbacks for using ZK for this configuration:

- Chukwa would be more complicated with no real benefit. Having a "central configuration system"
 was what Flume old generation did. In the new generation, they decided to get rid of it.
 On the whole, I think that Chukwa should inspire from the Linux philosophy: just do one think
 and do it (very) well. Such central configuration problem could be solved by Puppet, Chef
or other, no?
 And I have no doubt that companies with "large scale system" use one.

- I don't really like the idea of the agent depending on more hadoop components. Usually,
when I
 have to install a new Hadoop system in a customer datacenter, Chukwa agent is the only piece
 of sofware I have to install on servers that are already in production. So I like to say
 to the client "Chukwa agent is very easy to install, has little impact on your system and
 has no dependencies apart from java 6". If the ZK system should be implemented, I believe
 it would be great if we could choose between this system or the current configuration method.

> Use ZooKeeper for collector list and adaptor check list
> -------------------------------------------------------
>                 Key: CHUKWA-670
>                 URL:
>             Project: Chukwa
>          Issue Type: New Feature
>          Components: Data Collection
>         Environment: Java, ZooKeeper, Chukwa
>            Reporter: Eric Yang
> Agent is currently using local disk to storage check point file, and collector list.
 This can be hard to manage for large scale system that needs to be updated more frequently.
 The proposal is to have a new configuration in chukwa agent configuration file to enable
ZooKeeper as the location to store check point file and collector list.

This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see:

View raw message