incubator-s4-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Matthieu Morel <mmo...@apache.org>
Subject Re: How can I run counter example in s4 piper
Date Wed, 13 Mar 2013 10:25:38 GMT
There seems to be a regression indeed, the zookeeper connection string is not handled well.
So we'll cancel the release candidate, fix reported issues, and propose a new release candidate.

Thanks for the feedback!

Matthieu


On Mar 13, 2013, at 10:50 , larryzhang wrote:

> Just correct one mistake, the command used to create cluster is './s4 newCluster -c=cluster1
... -zk=<zkserver ip>.', thanks.
> On 03/13/2013 05:45 PM, larryzhang wrote:
>> Hi Morel,
>>      I download the 0.6.0 version and followed the document. However, if I use standalone
zookeeper, I can start cluster with the command './s4 addCluster -c=cluser1 ... -zk=<zkserver
ip>' , but can't add node with the command './s4 node -c=cluster1 -zk=<zkserver ip>',
the error info shows me that it can't connect to zookeeper server(timeout). 
>>     However, if I change back to 0.5.0 version, everything works well. Did I miss
something or it's  an issue?
>> Thanks,
>> Larry
>> 
>> On 03/13/2013 01:29 AM, Matthieu Morel wrote:
>>> (For some reason my answer sent through apache's stmp does not seem to reach
destination. So here it is again - ignore the other mail if it finally gets delivered)
>>> 
>>> 
>>> Hi Larry,
>>> 
>>> If you are totally new to S4, you are right on time to use the latest version.
We are currently voting on a first release candidate for S4 0.6.0, which is available there:
http://people.apache.org/~mmorel/s4-0.6.0-incubating-release-candidate-1/
>>> with updated documentation (still a few updates to add) here: http://incubator.apache.org/s4/doc/0.6.0
>>> 
>>> I'll answer your questions inline and refer to the 0.6 release when appropriate.
>>> 
>>> 
>>> On Mar 12, 2013, at 11:49 , larryzhang wrote:
>>> 
>>>> Hi,
>>>>    I'm totally new to S4, and want to run the counter example in 'apache-s4-...-source/subprojects/s4-examples/org/apache/s4/example/counter',
I have compiled all codes using gradle, and also type 'gradle eclipse' under the project root
folder 'apache-s4-...source/'. Then I import all subprojects to eclipse. 
>>>>    I found most of the codes in this example is easy to understand, except
'Module.java', I didn't know the following things:
>>>>      a) in function 'loadProperties':  
>>>>            InputStream is = this.getClass().getResourceAsStream("/s4-piper-example.properties");
 
>>>>              Q1: where is the file 's4-piper-example.properties', and what
does it used for? I searched web and find these contents:
>>>>                       --------------
>>>>                       pe.counter.interval = 1
>>>>                       comm.queue_emmiter_size = 8000
>>>>                       comm.queue_listener_size = 8000
>>>>                       cluster.hosts = localhost
>>>>                       cluster.ports = 5077
>>>>                       cluster.lock_dir = /tmp
>>>>                      ---------------
>>>>                 Is that main the cluster environment must be set up firstly?
>>> 
>>> The example is not maintained, even for 0.5.0, see https://github.com/apache/incubator-s4/blob/0.5.0/subprojects/s4-example/src/main/java/org/apache/s4/example/counter/README.md
>>> 
>>>>        
>>>>      b) In function 'configure', all codes are commented, e.g:
>>>>             ----------------
>>>>              // bind(MyApp.class);
>>>>             //
>>>>             // bind(PhysicalCluster.class);
>>>>            //
>>>>            // /* Configure static assignment using a configuration file.
*/
>>>>            // bind(Assignment.class).to(AssignmentFromFile.class);
>>>>            ------------------
>>>>            Q2: shall I need to change something in this function? I note
that 'Emitter and Listener' need to be bind, I thought these are like 'adapter' which I saw
in the walkthroughhttps://cwiki.apache.org/confluence/display/S4/S4+piper+walkthrough, which
are used to define source, am I right?
>>> 
>>> You may ignore this commented code. We are currently adding clarifications on
the ways to inject events into S4 0.6 in the website documentation (i.e. through adapters
or directly from the App class)
>>> 
>>>> 
>>>>       c) I note MyApp.java has main class, so I run it directly in eclipse,
and then got lots of errors occurs:
>>>>                     
>>>>      ---------------------------------------------------------------------------------------------
>>>> exception in thread "main" com.google.inject.ConfigurationException: Guice
configuration errors:
>>>> 1) No implementation for java.lang.Integer annotated with @com.google.inject.name.Named(value=s4.cluster.zk_connection_timeout)
was bound.
>>>>   while locating java.lang.Integer annotated with @com.google.inject.name.Named(value=s4.cluster.zk_connection_timeout)
>>>>     for parameter 2 at org.apache.s4.comm.topology.RemoteStreams.<init>(RemoteStreams.java:92)
>>>>   while locating org.apache.s4.comm.topology.RemoteStreams
>>>>     for field at org.apache.s4.core.RemoteSenders.streams(RemoteSenders.java:43)
>>>>   while locating org.apache.s4.core.RemoteSenders
>>>>     for field at org.apache.s4.core.App.remoteSenders(App.java:47)
>>>>   while locating org.apache.s4.example.counter.MyApp
>>>> 
>>>> 2) No implementation for java.lang.Integer annotated with @com.google.inject.name.Named(value=s4.cluster.zk_connection_timeout)
was bound.
>>>>   while locating java.lang.Integer annotated with @com.google.inject.name.Named(value=s4.cluster.zk_connection_timeout)
>>>>     for parameter 2 at org.apache.s4.comm.topology.RemoteStreams.<init>(RemoteStreams.java:92)
>>>>   while locating org.apache.s4.comm.topology.RemoteStreams
>>>>     for field at org.apache.s4.core.App.remoteStreams(App.java:47)
>>>>   while locating org.apache.s4.example.counter.MyApp
>>>> 
>>>>    ....  ....
>>>> 7) No implementation for org.apache.s4.base.Emitter was bound.
>>>>   while locating org.apache.s4.base.Emitter
>>>>     for parameter 0 at org.apache.s4.core.Sender.<init>(Sender.java:62)
>>>>   while locating org.apache.s4.core.Sender
>>>>     for field at org.apache.s4.core.App.sender(App.java:47)
>>>>   while locating org.apache.s4.example.counter.MyApp
>>>> 8) No implementation for org.apache.s4.base.Hasher was bound.
>>>>   while locating org.apache.s4.base.Hasher
>>>>     for parameter 2 at org.apache.s4.core.Sender.<init>(Sender.java:62)
>>>>   while locating org.apache.s4.core.Sender
>>>>     for field at org.apache.s4.core.App.sender(App.java:47)
>>>>   while locating org.apache.s4.example.counter.MyApp
>>>>    ...   ....
>>>> ------------------------------------------------------------------------------------------------------------------------
>>>>     Q3: I know these errors are related to my above questions, how can I
handle these?
>>> 
>>> You cannot run an S4 app independently of S4. See http://incubator.apache.org/s4/doc/0.6.0/dev_tips/
for how to do that. Basically you need to upload the application configuration to the cluster
manager first, then start the node.
>>> 
>>>>     Q4: One more thing, in main function, a Sender and a Receiver are created,
which confuse me because I didn't see these in walkthrough page, what does these mean?
>>>>           -----------------------------
>>>>        Sender sender = injector.getInstance(Sender.class);
>>>>         Receiver receiver = injector.getInstance(Receiver.class);
>>>>          ----------------------------- 
>>>>  
>>> 
>>> Again, you are referring the the counter app which is not maintained. I recommend
you look at the walkthrough and the twitter app for a more complete example.
>>> 
>>> 
>>> Hope this helps,
>>> 
>>> Matthieu
>>> 
>>> On Mar 12, 2013, at 11:49 , larryzhang wrote:
>>> 
>>>> Hi,
>>>>    I'm totally new to S4, and want to run the counter example in 'apache-s4-...-source/subprojects/s4-examples/org/apache/s4/example/counter',
I have compiled all codes using gradle, and also type 'gradle eclipse' under the project root
folder 'apache-s4-...source/'. Then I import all subprojects to eclipse. 
>>>>    I found most of the codes in this example is easy to understand, except
'Module.java', I didn't know the following things:
>>>>      a) in function 'loadProperties':  
>>>>            InputStream is = this.getClass().getResourceAsStream("/s4-piper-example.properties");
 
>>>>              Q1: where is the file 's4-piper-example.properties', and what
does it used for? I searched web and find these contents:
>>>>                       --------------
>>>>                       pe.counter.interval = 1
>>>>                       comm.queue_emmiter_size = 8000
>>>>                       comm.queue_listener_size = 8000
>>>>                       cluster.hosts = localhost
>>>>                       cluster.ports = 5077
>>>>                       cluster.lock_dir = /tmp
>>>>                      ---------------
>>>>                 Is that main the cluster environment must be set up firstly?
>>>>        
>>>>      b) In function 'configure', all codes are commented, e.g:
>>>>             ----------------
>>>>              // bind(MyApp.class);
>>>>             //
>>>>             // bind(PhysicalCluster.class);
>>>>            //
>>>>            // /* Configure static assignment using a configuration file.
*/
>>>>            // bind(Assignment.class).to(AssignmentFromFile.class);
>>>>            ------------------
>>>>            Q2: shall I need to change something in this function? I note
that 'Emitter and Listener' need to be bind, I thought these are like 'adapter' which I saw
in the walkthrough https://cwiki.apache.org/confluence/display/S4/S4+piper+walkthrough, which
are used to define source, am I right?
>>>> 
>>>>       c) I note MyApp.java has main class, so I run it directly in eclipse,
and then got lots of errors occurs:
>>>>                     
>>>>      ---------------------------------------------------------------------------------------------
>>>> exception in thread "main" com.google.inject.ConfigurationException: Guice
configuration errors:
>>>> 1) No implementation for java.lang.Integer annotated with @com.google.inject.name.Named(value=s4.cluster.zk_connection_timeout)
was bound.
>>>>   while locating java.lang.Integer annotated with @com.google.inject.name.Named(value=s4.cluster.zk_connection_timeout)
>>>>     for parameter 2 at org.apache.s4.comm.topology.RemoteStreams.<init>(RemoteStreams.java:92)
>>>>   while locating org.apache.s4.comm.topology.RemoteStreams
>>>>     for field at org.apache.s4.core.RemoteSenders.streams(RemoteSenders.java:43)
>>>>   while locating org.apache.s4.core.RemoteSenders
>>>>     for field at org.apache.s4.core.App.remoteSenders(App.java:47)
>>>>   while locating org.apache.s4.example.counter.MyApp
>>>> 
>>>> 2) No implementation for java.lang.Integer annotated with @com.google.inject.name.Named(value=s4.cluster.zk_connection_timeout)
was bound.
>>>>   while locating java.lang.Integer annotated with @com.google.inject.name.Named(value=s4.cluster.zk_connection_timeout)
>>>>     for parameter 2 at org.apache.s4.comm.topology.RemoteStreams.<init>(RemoteStreams.java:92)
>>>>   while locating org.apache.s4.comm.topology.RemoteStreams
>>>>     for field at org.apache.s4.core.App.remoteStreams(App.java:47)
>>>>   while locating org.apache.s4.example.counter.MyApp
>>>> 
>>>>    ....  ....
>>>> 7) No implementation for org.apache.s4.base.Emitter was bound.
>>>>   while locating org.apache.s4.base.Emitter
>>>>     for parameter 0 at org.apache.s4.core.Sender.<init>(Sender.java:62)
>>>>   while locating org.apache.s4.core.Sender
>>>>     for field at org.apache.s4.core.App.sender(App.java:47)
>>>>   while locating org.apache.s4.example.counter.MyApp
>>>> 8) No implementation for org.apache.s4.base.Hasher was bound.
>>>>   while locating org.apache.s4.base.Hasher
>>>>     for parameter 2 at org.apache.s4.core.Sender.<init>(Sender.java:62)
>>>>   while locating org.apache.s4.core.Sender
>>>>     for field at org.apache.s4.core.App.sender(App.java:47)
>>>>   while locating org.apache.s4.example.counter.MyApp
>>>>    ...   ....
>>>> ------------------------------------------------------------------------------------------------------------------------
>>>>     Q3: I know these errors are related to my above questions, how can I
handle these?
>>>>     Q4: One more thing, in main function, a Sender and a Receiver are created,
which confuse me because I didn't see these in walkthrough page, what does these mean?
>>>>           -----------------------------
>>>>        Sender sender = injector.getInstance(Sender.class);
>>>>         Receiver receiver = injector.getInstance(Receiver.class);
>>>>          ----------------------------- 
>>>>  
>>>>    I'll appreciate any help you give, thanks a lot!
>>>> Regards,
>>>> larry 
>>>>               
>>> 
>> 
> 


Mime
View raw message