chukwa-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Bill Graham <>
Subject Cluster-specific Adaptors
Date Mon, 20 Sep 2010 21:58:05 GMT

In CHUKWA-515 we discussed the possibility being able to add an
adaptor bound to a given cluster:

I can actually see this being useful, especially now that it's easier
to add/remove agents with the Adaptor REST API. Looking into the code
it doesn't seem like it would be that hard to do, but I want to make
sure I'm not overlooking anything.

It seems like we could support this with a few small changes:

- Add the concept of tags to the Adaptor interface.
- AbstractAdator would support a getTags method which would return the
union of tags set on the Adaptor and the default tags on the
- Internal tag implementations on each would change to store tags in
maps, instead of concat'ed strings. This would allow for a "last in
wins" type of functionality so tags could be overriden. This assumes
of course that there should never be more than one of the same tag key
value, which I _assume_ is the case.
- The ChunkImpl constructor will call getTags on the agent, instead of
getDefaultTags the data factory.

The trickiest part as I see it is figuring out how to change the add
adaptor string syntax in ChukwaAgetn.processAddCommandE in a way that
both makes sense and doesn't break things. In it's current form it
doesn't have room for easy expansion except at the end of the line:

add [name =] <adaptor_class_name> <datatype> <adaptor specific params>
<initial offset>

Any thoughts or suggestions? There's also a potential gotcha with all
the impls of Adaptor.parseArgs either breaking or needing to


View raw message