incubator-s4-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Matthieu Morel <mmo...@apache.org>
Subject Re: How can I run counter example in s4 piper
Date Wed, 13 Mar 2013 16:35:12 GMT
For info, we have a new release candidate that fixes the issue you reported. It's available
here and we are currently voting on it in the s4-dev list http://people.apache.org/~mmorel/s4-0.6.0-incubating-release-candidate-2/

Matthieu

On Mar 13, 2013, at 11:25 , Matthieu Morel wrote:

> There seems to be a regression indeed, the zookeeper connection string is not handled
well. So we'll cancel the release candidate, fix reported issues, and propose a new release
candidate.
> 
> Thanks for the feedback!
> 
> Matthieu
> 
> 
> On Mar 13, 2013, at 10:50 , larryzhang wrote:
> 
>> Just correct one mistake, the command used to create cluster is './s4 newCluster
-c=cluster1 ... -zk=<zkserver ip>.', thanks.
>> On 03/13/2013 05:45 PM, larryzhang wrote:
>>> Hi Morel,
>>>      I download the 0.6.0 version and followed the document. However, if I use
standalone zookeeper, I can start cluster with the command './s4 addCluster -c=cluser1 ...
-zk=<zkserver ip>' , but can't add node with the command './s4 node -c=cluster1 -zk=<zkserver
ip>', the error info shows me that it can't connect to zookeeper server(timeout). 
>>>     However, if I change back to 0.5.0 version, everything works well. Did I
miss something or it's  an issue?
>>> Thanks,
>>> Larry
>>> 
>>> On 03/13/2013 01:29 AM, Matthieu Morel wrote:
>>>> (For some reason my answer sent through apache's stmp does not seem to reach
destination. So here it is again - ignore the other mail if it finally gets delivered)
>>>> 
>>>> 
>>>> Hi Larry,
>>>> 
>>>> If you are totally new to S4, you are right on time to use the latest version.
We are currently voting on a first release candidate for S4 0.6.0, which is available there:
http://people.apache.org/~mmorel/s4-0.6.0-incubating-release-candidate-1/
>>>> with updated documentation (still a few updates to add) here: http://incubator.apache.org/s4/doc/0.6.0
>>>> 
>>>> I'll answer your questions inline and refer to the 0.6 release when appropriate.
>>>> 
>>>> 
>>>> On Mar 12, 2013, at 11:49 , larryzhang wrote:
>>>> 
>>>>> Hi,
>>>>>    I'm totally new to S4, and want to run the counter example in 'apache-s4-...-source/subprojects/s4-examples/org/apache/s4/example/counter',
I have compiled all codes using gradle, and also type 'gradle eclipse' under the project root
folder 'apache-s4-...source/'. Then I import all subprojects to eclipse. 
>>>>>    I found most of the codes in this example is easy to understand, except
'Module.java', I didn't know the following things:
>>>>>      a) in function 'loadProperties':  
>>>>>            InputStream is = this.getClass().getResourceAsStream("/s4-piper-example.properties");
 
>>>>>              Q1: where is the file 's4-piper-example.properties', and
what does it used for? I searched web and find these contents:
>>>>>                       --------------
>>>>>                       pe.counter.interval = 1
>>>>>                       comm.queue_emmiter_size = 8000
>>>>>                       comm.queue_listener_size = 8000
>>>>>                       cluster.hosts = localhost
>>>>>                       cluster.ports = 5077
>>>>>                       cluster.lock_dir = /tmp
>>>>>                      ---------------
>>>>>                 Is that main the cluster environment must be set up firstly?
>>>> 
>>>> The example is not maintained, even for 0.5.0, see https://github.com/apache/incubator-s4/blob/0.5.0/subprojects/s4-example/src/main/java/org/apache/s4/example/counter/README.md
>>>> 
>>>>>        
>>>>>      b) In function 'configure', all codes are commented, e.g:
>>>>>             ----------------
>>>>>              // bind(MyApp.class);
>>>>>             //
>>>>>             // bind(PhysicalCluster.class);
>>>>>            //
>>>>>            // /* Configure static assignment using a configuration file.
*/
>>>>>            // bind(Assignment.class).to(AssignmentFromFile.class);
>>>>>            ------------------
>>>>>            Q2: shall I need to change something in this function? I note
that 'Emitter and Listener' need to be bind, I thought these are like 'adapter' which I saw
in the walkthroughhttps://cwiki.apache.org/confluence/display/S4/S4+piper+walkthrough, which
are used to define source, am I right?
>>>> 
>>>> You may ignore this commented code. We are currently adding clarifications
on the ways to inject events into S4 0.6 in the website documentation (i.e. through adapters
or directly from the App class)
>>>> 
>>>>> 
>>>>>       c) I note MyApp.java has main class, so I run it directly in eclipse,
and then got lots of errors occurs:
>>>>>                     
>>>>>      ---------------------------------------------------------------------------------------------
>>>>> exception in thread "main" com.google.inject.ConfigurationException:
Guice configuration errors:
>>>>> 1) No implementation for java.lang.Integer annotated with @com.google.inject.name.Named(value=s4.cluster.zk_connection_timeout)
was bound.
>>>>>   while locating java.lang.Integer annotated with @com.google.inject.name.Named(value=s4.cluster.zk_connection_timeout)
>>>>>     for parameter 2 at org.apache.s4.comm.topology.RemoteStreams.<init>(RemoteStreams.java:92)
>>>>>   while locating org.apache.s4.comm.topology.RemoteStreams
>>>>>     for field at org.apache.s4.core.RemoteSenders.streams(RemoteSenders.java:43)
>>>>>   while locating org.apache.s4.core.RemoteSenders
>>>>>     for field at org.apache.s4.core.App.remoteSenders(App.java:47)
>>>>>   while locating org.apache.s4.example.counter.MyApp
>>>>> 
>>>>> 2) No implementation for java.lang.Integer annotated with @com.google.inject.name.Named(value=s4.cluster.zk_connection_timeout)
was bound.
>>>>>   while locating java.lang.Integer annotated with @com.google.inject.name.Named(value=s4.cluster.zk_connection_timeout)
>>>>>     for parameter 2 at org.apache.s4.comm.topology.RemoteStreams.<init>(RemoteStreams.java:92)
>>>>>   while locating org.apache.s4.comm.topology.RemoteStreams
>>>>>     for field at org.apache.s4.core.App.remoteStreams(App.java:47)
>>>>>   while locating org.apache.s4.example.counter.MyApp
>>>>> 
>>>>>    ....  ....
>>>>> 7) No implementation for org.apache.s4.base.Emitter was bound.
>>>>>   while locating org.apache.s4.base.Emitter
>>>>>     for parameter 0 at org.apache.s4.core.Sender.<init>(Sender.java:62)
>>>>>   while locating org.apache.s4.core.Sender
>>>>>     for field at org.apache.s4.core.App.sender(App.java:47)
>>>>>   while locating org.apache.s4.example.counter.MyApp
>>>>> 8) No implementation for org.apache.s4.base.Hasher was bound.
>>>>>   while locating org.apache.s4.base.Hasher
>>>>>     for parameter 2 at org.apache.s4.core.Sender.<init>(Sender.java:62)
>>>>>   while locating org.apache.s4.core.Sender
>>>>>     for field at org.apache.s4.core.App.sender(App.java:47)
>>>>>   while locating org.apache.s4.example.counter.MyApp
>>>>>    ...   ....
>>>>> ------------------------------------------------------------------------------------------------------------------------
>>>>>     Q3: I know these errors are related to my above questions, how can
I handle these?
>>>> 
>>>> You cannot run an S4 app independently of S4. See http://incubator.apache.org/s4/doc/0.6.0/dev_tips/
for how to do that. Basically you need to upload the application configuration to the cluster
manager first, then start the node.
>>>> 
>>>>>     Q4: One more thing, in main function, a Sender and a Receiver are
created, which confuse me because I didn't see these in walkthrough page, what does these
mean?
>>>>>           -----------------------------
>>>>>        Sender sender = injector.getInstance(Sender.class);
>>>>>         Receiver receiver = injector.getInstance(Receiver.class);
>>>>>          ----------------------------- 
>>>>>  
>>>> 
>>>> Again, you are referring the the counter app which is not maintained. I recommend
you look at the walkthrough and the twitter app for a more complete example.
>>>> 
>>>> 
>>>> Hope this helps,
>>>> 
>>>> Matthieu
>>>> 
>>>> On Mar 12, 2013, at 11:49 , larryzhang wrote:
>>>> 
>>>>> Hi,
>>>>>    I'm totally new to S4, and want to run the counter example in 'apache-s4-...-source/subprojects/s4-examples/org/apache/s4/example/counter',
I have compiled all codes using gradle, and also type 'gradle eclipse' under the project root
folder 'apache-s4-...source/'. Then I import all subprojects to eclipse. 
>>>>>    I found most of the codes in this example is easy to understand, except
'Module.java', I didn't know the following things:
>>>>>      a) in function 'loadProperties':  
>>>>>            InputStream is = this.getClass().getResourceAsStream("/s4-piper-example.properties");
 
>>>>>              Q1: where is the file 's4-piper-example.properties', and
what does it used for? I searched web and find these contents:
>>>>>                       --------------
>>>>>                       pe.counter.interval = 1
>>>>>                       comm.queue_emmiter_size = 8000
>>>>>                       comm.queue_listener_size = 8000
>>>>>                       cluster.hosts = localhost
>>>>>                       cluster.ports = 5077
>>>>>                       cluster.lock_dir = /tmp
>>>>>                      ---------------
>>>>>                 Is that main the cluster environment must be set up firstly?
>>>>>        
>>>>>      b) In function 'configure', all codes are commented, e.g:
>>>>>             ----------------
>>>>>              // bind(MyApp.class);
>>>>>             //
>>>>>             // bind(PhysicalCluster.class);
>>>>>            //
>>>>>            // /* Configure static assignment using a configuration file.
*/
>>>>>            // bind(Assignment.class).to(AssignmentFromFile.class);
>>>>>            ------------------
>>>>>            Q2: shall I need to change something in this function? I note
that 'Emitter and Listener' need to be bind, I thought these are like 'adapter' which I saw
in the walkthrough https://cwiki.apache.org/confluence/display/S4/S4+piper+walkthrough, which
are used to define source, am I right?
>>>>> 
>>>>>       c) I note MyApp.java has main class, so I run it directly in eclipse,
and then got lots of errors occurs:
>>>>>                     
>>>>>      ---------------------------------------------------------------------------------------------
>>>>> exception in thread "main" com.google.inject.ConfigurationException:
Guice configuration errors:
>>>>> 1) No implementation for java.lang.Integer annotated with @com.google.inject.name.Named(value=s4.cluster.zk_connection_timeout)
was bound.
>>>>>   while locating java.lang.Integer annotated with @com.google.inject.name.Named(value=s4.cluster.zk_connection_timeout)
>>>>>     for parameter 2 at org.apache.s4.comm.topology.RemoteStreams.<init>(RemoteStreams.java:92)
>>>>>   while locating org.apache.s4.comm.topology.RemoteStreams
>>>>>     for field at org.apache.s4.core.RemoteSenders.streams(RemoteSenders.java:43)
>>>>>   while locating org.apache.s4.core.RemoteSenders
>>>>>     for field at org.apache.s4.core.App.remoteSenders(App.java:47)
>>>>>   while locating org.apache.s4.example.counter.MyApp
>>>>> 
>>>>> 2) No implementation for java.lang.Integer annotated with @com.google.inject.name.Named(value=s4.cluster.zk_connection_timeout)
was bound.
>>>>>   while locating java.lang.Integer annotated with @com.google.inject.name.Named(value=s4.cluster.zk_connection_timeout)
>>>>>     for parameter 2 at org.apache.s4.comm.topology.RemoteStreams.<init>(RemoteStreams.java:92)
>>>>>   while locating org.apache.s4.comm.topology.RemoteStreams
>>>>>     for field at org.apache.s4.core.App.remoteStreams(App.java:47)
>>>>>   while locating org.apache.s4.example.counter.MyApp
>>>>> 
>>>>>    ....  ....
>>>>> 7) No implementation for org.apache.s4.base.Emitter was bound.
>>>>>   while locating org.apache.s4.base.Emitter
>>>>>     for parameter 0 at org.apache.s4.core.Sender.<init>(Sender.java:62)
>>>>>   while locating org.apache.s4.core.Sender
>>>>>     for field at org.apache.s4.core.App.sender(App.java:47)
>>>>>   while locating org.apache.s4.example.counter.MyApp
>>>>> 8) No implementation for org.apache.s4.base.Hasher was bound.
>>>>>   while locating org.apache.s4.base.Hasher
>>>>>     for parameter 2 at org.apache.s4.core.Sender.<init>(Sender.java:62)
>>>>>   while locating org.apache.s4.core.Sender
>>>>>     for field at org.apache.s4.core.App.sender(App.java:47)
>>>>>   while locating org.apache.s4.example.counter.MyApp
>>>>>    ...   ....
>>>>> ------------------------------------------------------------------------------------------------------------------------
>>>>>     Q3: I know these errors are related to my above questions, how can
I handle these?
>>>>>     Q4: One more thing, in main function, a Sender and a Receiver are
created, which confuse me because I didn't see these in walkthrough page, what does these
mean?
>>>>>           -----------------------------
>>>>>        Sender sender = injector.getInstance(Sender.class);
>>>>>         Receiver receiver = injector.getInstance(Receiver.class);
>>>>>          ----------------------------- 
>>>>>  
>>>>>    I'll appreciate any help you give, thanks a lot!
>>>>> Regards,
>>>>> larry 
>>>>>               
>>>> 
>>> 
>> 
> 


Mime
View raw message