chukwa-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Ariel Rabkin <asrab...@gmail.com>
Subject Re: Multiple agents and multiple clusters
Date Mon, 18 Jul 2011 23:36:26 GMT
I think that's not so hard to do.

My instinct would be to have a custom connector with two vanilla
senders in it. You'll just catch the exception when it propagates out
of the sender, and then periodically try re-initializing the sender.

--Ari

On Thu, Jul 14, 2011 at 8:45 PM, highpointe <highpointe3i@gmail.com> wrote:
> Ahhhh. Good insight. I didn't think about that.
>
> Yes, I am aggregating the same data for two clusters.
>
> So now my challenge is:  If one cluster is unreachable...  How do I that cluster's
sender fail without affecting the other sender or the connector.
>
> Excellent insight Ari!  Thank you!
>
> Sent from my iPhone
>
> On Jul 14, 2011, at 11:54 AM, Ariel Rabkin <asrabkin@gmail.com> wrote:
>
>> Just to be clear -- you want the same data to end up in each cluster?
>>
>> You'll have to do some hacking. I think it won't be so very monstrous.
>> In the Agent, there's a notion of pluggable Connectors and Senders. I
>> think you want a new Connector with two senders.
>>
>> One caveat -- you'll need to think through what happens if just one
>> cluster is down, and perhaps other partial failure scenarios too.
>>
>> --Ari
>>
>> On Thu, Jul 14, 2011 at 10:44 AM, high pointe <highpointe3i@gmail.com> wrote:
>>> I have a situation where I need to take the one data stream, send it through
>>> two agents and have the end point residing in separate clusters.
>>> Rough explanation....
>>> Streaming Data Box ---> Chukwa Box
>>> Agent one on Chukwa Box sends feed to cluster one.
>>> Agent two on Chukwa box sends feed to cluster two.
>>> Does anyone know of a way to do this without hacking this baby to shreds?
>>>
>>> Thanks!
>>
>>
>>
>> --
>> Ari Rabkin asrabkin@gmail.com
>> UC Berkeley Computer Science Department
>



-- 
Ari Rabkin asrabkin@gmail.com
UC Berkeley Computer Science Department

Mime
View raw message