storm-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Lasantha Fernando <lasantha....@gmail.com>
Subject Re: Replace Netty with ZeroMQ in Storm 0.9.1
Date Wed, 30 Apr 2014 05:09:55 GMT
Hi all,

Stumbled upon this thread when trying to switch to zeromq and came across
this error after putting the storm-0mq to <STORM_HOME>/lib directory.

java.lang.ExceptionInInitializerError: null
at java.lang.Class.forName0(Native Method) ~[na:1.6.0_45]
at java.lang.Class.forName(Class.java:249) ~[na:1.6.0_45]
at clojure.lang.RT.loadClassForName(RT.java:2056) ~[clojure-1.4.0.jar:na]
at clojure.lang.RT.load(RT.java:419) ~[clojure-1.4.0.jar:na]
at clojure.lang.RT.load(RT.java:400) ~[clojure-1.4.0.jar:na]
at clojure.core$load$fn__4890.invoke(core.clj:5415) ~[clojure-1.4.0.jar:na]
at clojure.core$load.doInvoke(core.clj:5414) ~[clojure-1.4.0.jar:na]
at clojure.lang.RestFn.invoke(RestFn.java:408) [clojure-1.4.0.jar:na]
at clojure.lang.Var.invoke(Var.java:415) ~[clojure-1.4.0.jar:na]
at backtype.storm.messaging.zmq.<clinit>(Unknown Source)
~[storm-0mq-0.9.0.jar:0.9.2-incubating-SNAPSHOT]
at java.lang.Class.forName0(Native Method) ~[na:1.6.0_45]
at java.lang.Class.forName(Class.java:171) ~[na:1.6.0_45]
at
backtype.storm.messaging.TransportFactory.makeContext(TransportFactory.java:38)
~[storm-core-0.9.2-incubating-SNAPSHOT.jar:0.9.2-incubating-SNAPSHOT]
at backtype.storm.daemon.worker$worker_data$fn__4263.invoke(worker.clj:185)
~[na:0.9.2-incubating-SNAPSHOT]
at backtype.storm.util$assoc_apply_self.invoke(util.clj:792)
~[storm-core-0.9.2-incubating-SNAPSHOT.jar:0.9.2-incubating-SNAPSHOT]
at backtype.storm.daemon.worker$worker_data.invoke(worker.clj:181)
~[storm-core-0.9.2-incubating-SNAPSHOT.jar:0.9.2-incubating-SNAPSHOT]
at
backtype.storm.daemon.worker$eval4413$exec_fn__1102__auto____4414.invoke(worker.clj:353)
~[na:0.9.2-incubating-SNAPSHOT]
at clojure.lang.AFn.applyToHelper(AFn.java:185) [clojure-1.4.0.jar:na]
at clojure.lang.AFn.applyTo(AFn.java:151) [clojure-1.4.0.jar:na]
at clojure.core$apply.invoke(core.clj:601) ~[clojure-1.4.0.jar:na]
at
backtype.storm.daemon.worker$eval4413$mk_worker__4469.doInvoke(worker.clj:344)
[na:0.9.2-incubating-SNAPSHOT]
at clojure.lang.RestFn.invoke(RestFn.java:512) [clojure-1.4.0.jar:na]
at backtype.storm.daemon.worker$_main.invoke(worker.clj:454)
[na:0.9.2-incubating-SNAPSHOT]
at clojure.lang.AFn.applyToHelper(AFn.java:172) [clojure-1.4.0.jar:na]
at clojure.lang.AFn.applyTo(AFn.java:151) [clojure-1.4.0.jar:na]
at backtype.storm.daemon.worker.main(Unknown Source)
[storm-core-0.9.2-incubating-SNAPSHOT.jar:0.9.2-incubating-SNAPSHOT]
Caused by: java.lang.ClassNotFoundException: org.zeromq.ZMQ
at java.net.URLClassLoader$1.run(URLClassLoader.java:202) ~[na:1.6.0_45]
at java.security.AccessController.doPrivileged(Native Method) ~[na:1.6.0_45]
at java.net.URLClassLoader.findClass(URLClassLoader.java:190) ~[na:1.6.0_45]
at java.lang.ClassLoader.loadClass(ClassLoader.java:306) ~[na:1.6.0_45]
at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:301)
~[na:1.6.0_45]
at java.lang.ClassLoader.loadClass(ClassLoader.java:247) ~[na:1.6.0_45]
at java.lang.Class.forName0(Native Method) ~[na:1.6.0_45]
at java.lang.Class.forName(Class.java:171) ~[na:1.6.0_45]
at backtype.storm.messaging.zmq$loading__4784__auto__.invoke(zmq.clj:1)
~[storm-0mq-0.9.0.jar:0.9.2-incubating-SNAPSHOT]
at backtype.storm.messaging.zmq__init.load(Unknown Source)
~[storm-0mq-0.9.0.jar:0.9.2-incubating-SNAPSHOT]
at backtype.storm.messaging.zmq__init.<clinit>(Unknown Source)
~[storm-0mq-0.9.0.jar:0.9.2-incubating-SNAPSHOT]
... 26 common frames omitted

This got resolved after copying 'zmq.jar' from /usr/local/share/java/ to
the lib directory as well. There might be other ways to resolve this issue
other than copying the 'zmq.jar'. Just shared in case it will be useful for
someone who comes across a similar issue.

Thanks,
Lasantha


On Mon, Apr 7, 2014 at 4:17 AM, Daria Mayorova <d.mayorova@gmail.com> wrote:

> Hi Neelesh,
>
> We are having some issues with the running topology, I described it here –
> https://www.mail-archive.com/user@storm.incubator.apache.org/msg01655.html
>
> We want to try using 0mq to check if replacing netty with it will make any
> change.
>
>
> On Sun, Apr 6, 2014 at 3:28 PM, Neelesh <neeleshs@gmail.com> wrote:
>
>> Hi Daria, I'm curious about why you decided to go back to zmq instead of
>> Netty. Did you hit any snags?
>>
>>
>> On Fri, Apr 4, 2014 at 6:34 AM, Daria Mayorova <d.mayorova@gmail.com>wrote:
>>
>>> Thanks for the info, Jon.
>>>
>>> You're right, I've found this repository –
>>> https://github.com/ptgoetz/storm-0mq. I guess this is what you meant.
>>> I'll give it a try, let's see if I can make it work.
>>>
>>>
>>> On Fri, Apr 4, 2014 at 2:58 PM, Jon Logan <jmlogan@buffalo.edu> wrote:
>>>
>>>> You need to include the zeromq Storm binding in your storm lib folder
>>>> probably...it's no longer packaged together due to GPL licensing issues.
>>>> I've never done this, but I think it was broken off into a separate github
>>>> repository.
>>>>
>>>>
>>>> On Fri, Apr 4, 2014 at 7:01 AM, Daria Mayorova <d.mayorova@gmail.com>wrote:
>>>>
>>>>> Thank you Susheel,
>>>>>
>>>>> We've tried to change this parameter, but the topology isn't
>>>>> initializing. The following exception is returned:
>>>>>
>>>>> 2014-04-04 10:35:28 b.s.d.worker [ERROR] Error on initialization of
>>>>> server mk-worker
>>>>> java.lang.RuntimeException: Fail to construct messaging plugin from
>>>>> plugin backtype.storm.messaging.zmq
>>>>>         at
>>>>> backtype.storm.messaging.TransportFactory.makeContext(TransportFactory.java:53)
>>>>> ~[storm-core-0.9.1-incubating.jar:0.9.1-incubating]
>>>>>         at
>>>>> backtype.storm.daemon.worker$worker_data$fn__4263.invoke(worker.clj:185)
>>>>> ~[na:0.9.1-incubating]
>>>>>         at backtype.storm.util$assoc_apply_self.invoke(util.clj:792)
>>>>> ~[storm-core-0.9.1-incubating.jar:0.9.1-incubating]
>>>>>         at
>>>>> backtype.storm.daemon.worker$worker_data.invoke(worker.clj:181)
>>>>> ~[storm-core-0.9.1-incubating.jar:0.9.1-incubating]
>>>>>         at
>>>>> backtype.storm.daemon.worker$eval4413$exec_fn__1102__auto____4414.invoke(worker.clj:353)
>>>>> ~[na:0.9.1-incubating]
>>>>>         at clojure.lang.AFn.applyToHelper(AFn.java:185)
>>>>> [clojure-1.4.0.jar:na]
>>>>>         at clojure.lang.AFn.applyTo(AFn.java:151)
>>>>> [clojure-1.4.0.jar:na]
>>>>>         at clojure.core$apply.invoke(core.clj:601)
>>>>> ~[clojure-1.4.0.jar:na]
>>>>>         at
>>>>> backtype.storm.daemon.worker$eval4413$mk_worker__4469.doInvoke(worker.clj:344)
>>>>> [na:0.9.1-incubating]
>>>>>         at clojure.lang.RestFn.invoke(RestFn.java:512)
>>>>> [clojure-1.4.0.jar:na]
>>>>>         at backtype.storm.daemon.worker$_main.invoke(worker.clj:454)
>>>>> [na:0.9.1-incubating]
>>>>>         at clojure.lang.AFn.applyToHelper(AFn.java:172)
>>>>> [clojure-1.4.0.jar:na]
>>>>>         at clojure.lang.AFn.applyTo(AFn.java:151)
>>>>> [clojure-1.4.0.jar:na]
>>>>>         at backtype.storm.daemon.worker.main(Unknown Source)
>>>>> [storm-core-0.9.1-incubating.jar:0.9.1-incubating]
>>>>> Caused by: java.lang.ClassNotFoundException:
>>>>> backtype.storm.messaging.zmq
>>>>>         at java.net.URLClassLoader$1.run(URLClassLoader.java:217)
>>>>> ~[na:1.6.0_27]
>>>>>         at java.security.AccessController.doPrivileged(Native Method)
>>>>> ~[na:1.6.0_27]
>>>>>         at java.net.URLClassLoader.findClass(URLClassLoader.java:205)
>>>>> ~[na:1.6.0_27]
>>>>>         at java.lang.ClassLoader.loadClass(ClassLoader.java:321)
>>>>> ~[na:1.6.0_27]
>>>>>         at
>>>>> sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:294) ~[na:1.6.0_27]
>>>>>         at java.lang.ClassLoader.loadClass(ClassLoader.java:266)
>>>>> ~[na:1.6.0_27]
>>>>>         at java.lang.Class.forName0(Native Method) ~[na:1.6.0_27]
>>>>>         at java.lang.Class.forName(Class.java:188) ~[na:1.6.0_27]
>>>>>         at
>>>>> backtype.storm.messaging.TransportFactory.makeContext(TransportFactory.java:38)
>>>>> ~[storm-core-0.9.1-incubating.jar:0.9.1-incubating]
>>>>>         ... 13 common frames omitted
>>>>> 2014-04-04 10:35:28 b.s.util [INFO] Halting process: ("Error on
>>>>> initialization")
>>>>>
>>>>> 0mq and jzmq are installed correctly on the machine, as before Storm
>>>>> 0.8.1 was running on it using 0mq.
>>>>>
>>>>> Any other thing that should be done to make this work?
>>>>>
>>>>>
>>>>> On Fri, Apr 4, 2014 at 10:52 AM, Susheel Kumar Gadalay <
>>>>> skgadalay@gmail.com> wrote:
>>>>>
>>>>>> Set this parameter in storm.yaml
>>>>>>
>>>>>> storm.messaging.transport: "backtype.storm.messaging.zmq"
>>>>>>
>>>>>> On 4/4/14, Daria Mayorova <d.mayorova@gmail.com> wrote:
>>>>>> > Hi everyone,
>>>>>> >
>>>>>> > I am wondering whether it is possible to go back to using 0mq
as
>>>>>> transport
>>>>>> > instead of Netty? I am pretty sure I saw some messages about
>>>>>> switching back
>>>>>> > to 0mq in Storm 0.9.1 at the old storm-user google group.
>>>>>> > There is an issue related to this on Jira -
>>>>>> > https://issues.apache.org/jira/browse/STORM-215 - but it seems
to
>>>>>> be
>>>>>> > abandoned.
>>>>>> >
>>>>>> > Can anybody give me an idea of how this can be achieved?
>>>>>> >
>>>>>> > Thanks.
>>>>>> >
>>>>>> > Best regards,
>>>>>> > Daria
>>>>>> >
>>>>>>
>>>>>
>>>>>
>>>>
>>>
>>
>

Mime
View raw message