storm-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Erik Weathers <eweath...@groupon.com>
Subject Re: storm.yaml config: change on nimbus or worker machines?
Date Fri, 05 Dec 2014 18:37:05 GMT
Thanks Susheel.  Do you know enough about how storm works internally to be
able to answer my other questions?  Specifically, I'd like to know how to
change the configuration back in the future; would I need to have the
topologies resubmitted in order to have the configuration passed along?  Or
could I just kill all the workers and when storm creates new workers it
would propagate the changed settings from the nimbus?

Also, which configuration options are ok to be in the worker host's
storm.yaml (since from my experiment it seemed insufficient to set this "
storm.messaging.transport" value in the worker's storm.yaml -- it *seemed*
this way because the nimbus UI was showing a different value than what I
configured in the worker storm.yaml)?  Where is the distinction drawn
between a config option that needs to be in the nimbus storm.yaml versus
the supervisor/worker storm.yaml (a pointer to the code would be awesome)?
I'm still confused about how configurations are handled in storm and I'm
worried that there could be different configurations in place on workers
versus what the nimbus shows.

- Erik

On Fri, Dec 5, 2014 at 12:32 AM, Susheel Kumar Gadalay <skgadalay@gmail.com>
wrote:

> If required globally for all topologies then set it in Nimbus.
>
> It will be carried to all workers when topology submitted.
>
> On 12/5/14, Erik Weathers <eweathers@groupon.com> wrote:
> > Thanks for the response Susheel.  But this setting is required for all
> > topologies, I need to set it globally so that I don't have to harass
> every
> > team into modifying their topologies with that configuration.  Also,
> later
> > when netty is presumably fixed I don't want to force people to rewrite
> > their topologies to switch back from zmq to netty.
> >
> > - Erik
> >
> > On Thu, Dec 4, 2014 at 10:08 PM, Susheel Kumar Gadalay
> > <skgadalay@gmail.com>
> > wrote:
> >
> >> If this is required for one topology better to set it in the Config
> >> object.
> >>
> >> Config config = new Config();
> >> config.put(Config.storm.messaging.transport:
> >> "backtype.storm.messaging.zmq");
> >>
> >>
> >> On 12/5/14, Erik Weathers <eweathers@groupon.com> wrote:
> >> > Can someone please clarify for me how you properly specify
> >> worker-specific
> >> > configuration in storm.yaml?
> >> >
> >> > e.g., I'm making a change to use zmq as the transport (due to a
> >> tangential
> >> > problem with netty), and it's unclear where to make this change:
> >> >
> >> >    - storm.messaging.transport: backtype.storm.messaging.zmq
> >> >
> >> > I assumed that I needed to change storm.yaml on the worker machine,
> but
> >> it
> >> > seems that the change only shows up in the nimbus UI when I modify
> >> > storm.yaml on the nimbus.
> >> >
> >> > So I'm assuming that the config options you specify in storm.yaml on
> >> nimbus
> >> > propagated to the storm workers, please correct me if that's wrong.
> >> >
> >> > But what if you make a change to the configs in the nimbus and then
> >> restart
> >> > it with the topology still running?  i.e., will that change be
> >> > propagated
> >> > to the existing topology workers somehow?  (In the case of changing
> the
> >> > messaging transport, I assume that would mean restarting the workers
> to
> >> use
> >> > the alternative transport layer.)
> >> >
> >> > Thanks!
> >> >
> >> > - Erik
> >> >
> >> > P.S., I am well aware of the other stuff you need to do to get zmq
> >> working
> >> > in storm-0.9.1+; I just don't want to force all of our topologies to
> >> > specify the above configuration knob.
> >> >
> >> > P.P.S., I'm using storm-0.9.3
> >> >
> >>
> >
>

Mime
View raw message