storm-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Eyal Golan <egola...@gmail.com>
Subject Re: Shared static data between bolts
Date Thu, 04 Dec 2014 20:34:03 GMT
Michael,
I understand your solution, however, if bolt of type A "wins" the
initialization, will bolts of type B see the same instance?

Sent from my Android phone
On Dec 4, 2014 10:26 PM, "Eyal Golan" <egolan74@gmail.com> wrote:

> Thanks Michael and Ben !
>
> Sent from my Android phone
> On Dec 4, 2014 9:53 PM, "Flint, Ben" <bflint@popcap.com> wrote:
>
>>  I’ve used Hazelcast for use cases like this with great success.  It
>> makes it relatively straightforward to implement a distributed data
>> structure.
>>
>>   From: Eyal Golan <egolan74@gmail.com>
>> Reply-To: "user@storm.apache.org" <user@storm.apache.org>
>> Date: Thursday, December 4, 2014 at 11:40 AM
>> To: "user@storm.apache.org" <user@storm.apache.org>
>> Subject: Re: Shared static data between bolts
>>
>>   Thanks!
>> All the answers give me good idea.
>> Here's why I need this list be sharable.
>> One bolt needs the list for computation.
>> Another bolt will periodically update the list.
>> In other words, the list is changeable.
>>
>> I tried passing it in conf but there are some 3rd party libs in the
>> wrapper.
>>
>> Anyway, I also need to improve the topology. Separate bolt to two etc.
>> That way, it would be easier to handle that cache.
>>
>> One more question, I thought about giving hazelcast a try.
>> Create local node and then the bolts will have clients.
>>
>> What do you think?
>>
>> Sent from my Android phone
>> On Dec 4, 2014 9:14 PM, "Kushan Maskey" <
>> kushan.maskey@mmillerassociates.com> wrote:
>>
>>>  Hi Eyal,
>>>
>>>  I had the similar issue earlier. Static objects are not a good way of
>>> passing the variables around in Storm due to clustered nature of the Storm.
>>> You will see null for all the static variables.
>>>
>>>  This is how I implemented mine, for instance I wanted to pass around
>>> the server properties such and host name and username/password. I read the
>>> properties file and then created a map of these properties then set it into
>>> the Config object. Like below.
>>>
>>>  *In topology:*
>>>
>>>  Map<Stirng, Stirng> map = new HashMap();
>>> map.put("hostname", "localhost");
>>>  map.put("username", "testuser");
>>>  map.put("password", "testpassord");
>>> map.put("port", "1234");
>>>
>>> Config config = new Config();
>>>
>>> config.put("MY_MAP", map)
>>>  *In Bolts:*
>>>
>>> public void prepare(Map stormConf, TopologyContext context) {
>>>
>>> HashMap<Stirng, Stirng> map = stormConfig.get("MY_MAP");
>>>
>>> }
>>>  This how i resolved my problem.Try that if that helps.
>>>   --
>>> Kushan Maskey
>>> 817.403.7500
>>> M. Miller & Associates <http://mmillerassociates.com/>
>>> kushan.maskey@mmillerassociates.com
>>>
>>> On Thu, Dec 4, 2014 at 1:01 PM, Eyal Golan <egolan74@gmail.com> wrote:
>>>
>>>> Hi all,
>>>> I am newbie with Storm and having some issues with static objects in
>>>> the topology.
>>>> In the main class, where I set the topology, I also set a static
>>>> object. It's actually a wrapper for a collection of strings.
>>>> This object needs to he shared between different bolts.
>>>> It's big so I want it to be only one instance.
>>>>
>>>> The problem is that when a bolt tries to get this static instance:
>>>> example - MyMain.Shared... It's null. Although I instantiated it in the
>>>> main method.
>>>>
>>>> I guess I'm doing something wrong.
>>>>
>>>> Thanks for any assistance.
>>>>
>>>> Eyal
>>>>
>>>> Sent from my Android phone
>>>>
>>>
>>>

Mime
View raw message