incubator-s4-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Matthieu Morel <matthieu.mo...@gmail.com>
Subject Re: Problems about Running Twitter Trending Example
Date Fri, 09 Nov 2012 11:15:26 GMT
Hi,

It looks like you are using the code in S4-25 branch, which was
slightly modified with respect to the release, and the wiki documents
the released version unless specified differently.

It was changed so that it does not rely on a "twitter4j.properties"
file in the root of your home dir, because that makes less sense for a
deployment on YARN.

You can fix the issue by starting the adapter with the following
inlined parameters (as in YARN example):
-p=s4.adapter.output.stream=RawStatus,twitter4j.debug=<true\|false>,twitter4j.user=<youUser>,twitter4j.password=<yourPassword>

Hope this helps,

Matthieu

On Fri, Nov 9, 2012 at 7:16 AM, Frank Zheng <bearzheng2011@gmail.com> wrote:
>
> Hi All,
>
> I tried to run the Twitter Trending Example in standalone mode.
> I followed step by step in the website "S4 piper walkthrough"
> After I deployed S4 applications, the adapter node had some errors as follows.
>
> [root@testing s4]# ./s4 node -c=cluster2 -p=s4.adapter.output.stream=RawStatus
> 14:09:36.304 [main] INFO  org.apache.s4.core.Main - processing inline configuration parameter
s4.adapter.output.stream=RawStatus
> 14:09:36.310 [main] INFO  org.apache.s4.core.Main - Initializing S4 node with :
> - comm module class [org.apache.s4.comm.DefaultCommModule]
> - comm configuration file [default.s4.comm.properties from classpath]
> - core module class [org.apache.s4.core.DefaultCoreModule]
> - core configuration file[default.s4.core.properties from classpath]
> - extra modules: []
> - inline parameters: [s4.adapter.output.stream=RawStatus]
> 14:09:36.317 [main] DEBUG org.apache.s4.core.Main - Adding named parameters for injection
: [s4.adapter.output.stream=RawStatus]
> 14:09:36.650 [main] INFO  org.apache.s4.core.Main - Starting S4 node. This node will
automatically download applications published for the cluster it belongs to
> 14:09:36.716 [main] INFO  o.a.s.comm.topology.AssignmentFromZK - New session:88639197229350934;
state is : SyncConnected
> 14:09:36.778 [main] INFO  o.a.s.comm.topology.AssignmentFromZK - Successfully acquired
task:Task-0 by testing.machine1
> 14:09:36.786 [main] DEBUG o.a.s4.deploy.FileSystemS4RFetcher - Fetching uri through the
file system : file:/tmp/1352527013477-0/twitter-adapter.s4r
> 14:09:36.806 [main] INFO  org.apache.s4.core.Server - Loading application [twitter-adapter]
from file [/tmp/tmp8036294422167861023s4r]
> 14:09:36.807 [main] WARN  o.a.s4.base.util.S4RLoaderFactory - s4.tmp.dir not specified,
using temporary directory [/tmp/1352527776806-0] for unpacking S4R. You may want to specify
a parent non-temporary directory.
> 14:09:36.807 [main] INFO  o.a.s4.base.util.S4RLoaderFactory - Unzipping S4R archive in
[/tmp/1352527776806-0]
> 14:09:36.902 [main] INFO  org.apache.s4.core.Server - App class name is: org.apache.s4.example.twitter.TwitterInputAdapter
> 14:09:36.963 [main] ERROR org.apache.s4.core.Server - Could not load s4 application form
s4r file [{/tmp/tmp8036294422167861023s4r}]
> com.google.inject.ConfigurationException: Guice configuration errors:
>
> 1) No implementation for java.lang.String annotated with @com.google.inject.name.Named(value=twitter4j.debug)
was bound.
>   while locating java.lang.String annotated with @com.google.inject.name.Named(value=twitter4j.debug)
>     for field at org.apache.s4.example.twitter.TwitterInputAdapter.twitter4jDebug(TwitterInputAdapter.java:43)
>   while locating org.apache.s4.example.twitter.TwitterInputAdapter
>
> 2) No implementation for java.lang.String annotated with @com.google.inject.name.Named(value=twitter4j.password)
was bound.
>   while locating java.lang.String annotated with @com.google.inject.name.Named(value=twitter4j.password)
>     for field at org.apache.s4.example.twitter.TwitterInputAdapter.twitterPassword(TwitterInputAdapter.java:43)
>   while locating org.apache.s4.example.twitter.TwitterInputAdapter
>
> 3) No implementation for java.lang.String annotated with @com.google.inject.name.Named(value=twitter4j.user)
was bound.
>   while locating java.lang.String annotated with @com.google.inject.name.Named(value=twitter4j.user)
>     for field at org.apache.s4.example.twitter.TwitterInputAdapter.twitterUser(TwitterInputAdapter.java:43)
>   while locating org.apache.s4.example.twitter.TwitterInputAdapter
>
> 3 errors
>     at com.google.inject.internal.InjectorImpl.getMembersInjector(InjectorImpl.java:952)
~[guice-3.0.jar:na]
>     at com.google.inject.internal.InjectorImpl.getMembersInjector(InjectorImpl.java:957)
~[guice-3.0.jar:na]
>     at com.google.inject.internal.InjectorImpl.injectMembers(InjectorImpl.java:943) ~[guice-3.0.jar:na]
>     at org.apache.s4.core.Server.loadApp(Server.java:120) [s4-core-0.5.0-incubating.jar:0.5.0-incubating]
>     at org.apache.s4.deploy.DistributedDeploymentManager.deployApplication(DistributedDeploymentManager.java:132)
[s4-core-0.5.0-incubating.jar:0.5.0-incubating]
>     at org.apache.s4.deploy.DistributedDeploymentManager.start(DistributedDeploymentManager.java:185)
[s4-core-0.5.0-incubating.jar:0.5.0-incubating]
>     at org.apache.s4.core.Server.start(Server.java:87) [s4-core-0.5.0-incubating.jar:0.5.0-incubating]
>     at org.apache.s4.core.Main.startNode(Main.java:159) [s4-core-0.5.0-incubating.jar:0.5.0-incubating]
>     at org.apache.s4.core.Main.main(Main.java:76) [s4-core-0.5.0-incubating.jar:0.5.0-incubating]
>     at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) ~[na:1.6.0_22]
>     at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
~[na:1.6.0_22]
>     at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
~[na:1.6.0_22]
>     at java.lang.reflect.Method.invoke(Method.java:616) ~[na:1.6.0_22]
>     at org.apache.s4.tools.Tools$Task.dispatch(Tools.java:54) [s4-tools-0.5.0-incubating.jar:0.5.0-incubating]
>     at org.apache.s4.tools.Tools.main(Tools.java:94) [s4-tools-0.5.0-incubating.jar:0.5.0-incubating]
> 14:09:36.963 [main] ERROR o.a.s.d.DistributedDeploymentManager - Cannot deploy application
> org.apache.s4.deploy.DeploymentFailedException: Cannot deploy application [twitter-adapter]
from URI [file:/tmp/1352527013477-0/twitter-adapter.s4r] : cannot start application
>     at org.apache.s4.deploy.DistributedDeploymentManager.deployApplication(DistributedDeploymentManager.java:138)
~[s4-core-0.5.0-incubating.jar:0.5.0-incubating]
>     at org.apache.s4.deploy.DistributedDeploymentManager.start(DistributedDeploymentManager.java:185)
~[s4-core-0.5.0-incubating.jar:0.5.0-incubating]
>     at org.apache.s4.core.Server.start(Server.java:87) [s4-core-0.5.0-incubating.jar:0.5.0-incubating]
>     at org.apache.s4.core.Main.startNode(Main.java:159) [s4-core-0.5.0-incubating.jar:0.5.0-incubating]
>     at org.apache.s4.core.Main.main(Main.java:76) [s4-core-0.5.0-incubating.jar:0.5.0-incubating]
>     at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) ~[na:1.6.0_22]
>     at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
~[na:1.6.0_22]
>     at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
~[na:1.6.0_22]
>     at java.lang.reflect.Method.invoke(Method.java:616) ~[na:1.6.0_22]
>     at org.apache.s4.tools.Tools$Task.dispatch(Tools.java:54) [s4-tools-0.5.0-incubating.jar:0.5.0-incubating]
>     at org.apache.s4.tools.Tools.main(Tools.java:94) [s4-tools-0.5.0-incubating.jar:0.5.0-incubating]
>
>
> And the counter nodes just halted and has the logs as follows:
>
> [root@testing s4]# ./s4 node -c=cluster1
> 13:54:51.754 [main] INFO  org.apache.s4.core.Main - Initializing S4 node with :
> - comm module class [org.apache.s4.comm.DefaultCommModule]
> - comm configuration file [default.s4.comm.properties from classpath]
> - core module class [org.apache.s4.core.DefaultCoreModule]
> - core configuration file[default.s4.core.properties from classpath]
> - extra modules: []
> - inline parameters: []
> 13:54:52.090 [main] INFO  org.apache.s4.core.Main - Starting S4 node. This node will
automatically download applications published for the cluster it belongs to
> 13:54:52.163 [main] INFO  o.a.s.comm.topology.AssignmentFromZK - New session:88639197229350917;
state is : SyncConnected
> 13:54:52.232 [main] INFO  o.a.s.comm.topology.AssignmentFromZK - Successfully acquired
task:Task-0 by testing.machine1
> 13:56:32.499 [ZkClient-EventThread-13-localhost:2181] DEBUG o.a.s4.deploy.FileSystemS4RFetcher
- Fetching uri through the file system : file:/tmp/1352526980763-0/twitter-counter.s4r
> 13:56:32.522 [ZkClient-EventThread-13-localhost:2181] INFO  org.apache.s4.core.Server
- Loading application [twitter-counter] from file [/tmp/tmp1410511865913040673s4r]
> 13:56:32.523 [ZkClient-EventThread-13-localhost:2181] WARN  o.a.s4.base.util.S4RLoaderFactory
- s4.tmp.dir not specified, using temporary directory [/tmp/1352526992522-0] for unpacking
S4R. You may want to specify a parent non-temporary directory.
> 13:56:32.523 [ZkClient-EventThread-13-localhost:2181] INFO  o.a.s4.base.util.S4RLoaderFactory
- Unzipping S4R archive in [/tmp/1352526992522-0]
> 13:56:32.673 [ZkClient-EventThread-13-localhost:2181] INFO  org.apache.s4.core.Server
- App class name is: org.apache.s4.example.twitter.TwitterCounterApp
> 13:56:32.772 [ZkClient-EventThread-13-localhost:2181] INFO  o.a.s4.comm.topology.ClusterFromZK
- Changing cluster topology to { nbNodes=2,name=cluster1,mode=unicast,type=,nodes=[{partition=0,port=12000,machineName=testing.machine1,taskId=Task-0},
{partition=1,port=12001,machineName=testing.machine1,taskId=Task-1}]} from null
> 13:56:32.813 [ZkClient-EventThread-13-localhost:2181] INFO  o.a.s4.comm.topology.ClusterFromZK
- Adding topology change listener:org.apache.s4.comm.tcp.TCPEmitter@38d30fb
> 13:56:32.857 [ZkClient-EventThread-13-localhost:2181] INFO  o.a.s4.comm.topology.ClustersFromZK
- New session:88639197229350929
> 13:56:32.873 [ZkClient-EventThread-13-localhost:2181] INFO  o.a.s4.comm.topology.ClustersFromZK
- New session:88639197229350930
> 13:56:32.890 [ZkClient-EventThread-13-localhost:2181] INFO  o.a.s4.comm.topology.ClusterFromZK
- Changing cluster topology to { nbNodes=2,name=cluster1,mode=unicast,type=,nodes=[{partition=0,port=12000,machineName=testing.machine1,taskId=Task-0},
{partition=1,port=12001,machineName=testing.machine1,taskId=Task-1}]} from null
> 13:56:32.898 [ZkClient-EventThread-13-localhost:2181] INFO  o.a.s4.comm.topology.ClusterFromZK
- Changing cluster topology to { nbNodes=1,name=cluster2,mode=unicast,type=,nodes=[{partition=0,port=13000,machineName=testing.machine1,taskId=Task-0}]}
from null
> 13:56:32.898 [ZkClient-EventThread-13-localhost:2181] INFO  org.apache.s4.core.Server
- Loaded application from file /tmp/tmp1410511865913040673s4r
> 13:56:32.898 [ZkClient-EventThread-13-localhost:2181] INFO  o.a.s.d.DistributedDeploymentManager
- Successfully installed application twitter-counter
> 13:56:32.921 [ZkClient-EventThread-13-localhost:2181] DEBUG o.a.s.c.g.OverloadDispatcherGenerator
- Dumping generated overload dispatcher class for PE of class [class org.apache.s4.example.twitter.TopNTopicPE]
> 13:56:32.940 [ZkClient-EventThread-24-localhost:2181] INFO  o.a.s4.comm.topology.ClustersFromZK
- Detected new stream [RawStatus]
> 13:56:32.943 [ZkClient-EventThread-13-localhost:2181] INFO  o.a.s4.example.twitter.TopNTopicPE
- key: []
> 13:56:32.949 [ZkClient-EventThread-13-localhost:2181] DEBUG o.a.s.c.g.OverloadDispatcherGenerator
- Dumping generated overload dispatcher class for PE of class [class org.apache.s4.example.twitter.TopicCountAndReportPE]
> 13:56:32.951 [ZkClient-EventThread-13-localhost:2181] DEBUG o.a.s.c.g.OverloadDispatcherGenerator
- Dumping generated overload dispatcher class for PE of class [class org.apache.s4.example.twitter.TopicExtractorPE]
> 13:56:32.956 [ZkClient-EventThread-13-localhost:2181] DEBUG o.a.s4.comm.topology.ClustersFromZK
- Adding input stream [RawStatus] for app [-1] in cluster [cluster1]
> 13:56:32.995 [ZkClient-EventThread-13-localhost:2181] INFO  org.apache.s4.core.App -
Init prototype [org.apache.s4.example.twitter.TopNTopicPE].
> 13:56:32.998 [ZkClient-EventThread-13-localhost:2181] DEBUG org.apache.s4.core.ProcessingElement
- Started timer for PE prototype [org.apache.s4.example.twitter.TopNTopicPE], ID [] with interval
[10000].
> 13:56:33.001 [ZkClient-EventThread-13-localhost:2181] DEBUG org.apache.s4.core.ProcessingElement
- Started checkpointing timer for PE prototype [org.apache.s4.example.twitter.TopNTopicPE],
ID [] with interval [20] [SECONDS].
> 13:56:33.002 [ZkClient-EventThread-13-localhost:2181] INFO  org.apache.s4.core.App -
Init prototype [org.apache.s4.example.twitter.TopicCountAndReportPE].
> 13:56:33.003 [ZkClient-EventThread-13-localhost:2181] DEBUG org.apache.s4.core.ProcessingElement
- Started timer for PE prototype [org.apache.s4.example.twitter.TopicCountAndReportPE], ID
[] with interval [10000].
> 13:56:33.003 [ZkClient-EventThread-13-localhost:2181] INFO  org.apache.s4.core.App -
Init prototype [org.apache.s4.example.twitter.TopicExtractorPE].
> 13:57:40.142 [ZkClient-EventThread-27-localhost:2181] INFO  o.a.s4.comm.topology.ClusterFromZK
- Changing cluster topology to { nbNodes=0,name=unknown,mode=unicast,type=,nodes=[]} from
{ nbNodes=1,name=cluster2,mode=unicast,type=,nodes=[{partition=0,port=13000,machineName=testing.machine1,taskId=Task-0}]}
> 14:09:36.780 [ZkClient-EventThread-27-localhost:2181] INFO  o.a.s4.comm.topology.ClusterFromZK
- Changing cluster topology to { nbNodes=1,name=cluster2,mode=unicast,type=,nodes=[{partition=0,port=13000,machineName=testing.machine1,taskId=Task-0}]}
from { nbNodes=0,name=unknown,mode=unicast,type=,nodes=[]}
>
>
> Could anyone tell me what's wrong with this?
>
> Thanks!
> Yu
>

Mime
View raw message