incubator-s4-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From larryzhang <zhangt...@gmail.com>
Subject Re: How can I run counter example in s4 piper
Date Wed, 13 Mar 2013 09:50:34 GMT
Just correct one mistake, the command used to create cluster is './s4 
newCluster -c=cluster1 ... -zk=<zkserver ip>.', thanks.
On 03/13/2013 05:45 PM, larryzhang wrote:
> Hi Morel,
>      I download the 0.6.0 version and followed the document. However, 
> if I use standalone zookeeper, I can start cluster with the command 
> './s4 addCluster -c=cluser1 ... -zk=<zkserver ip>' , but can't add 
> node with the command './s4 node -c=cluster1 -zk=<zkserver ip>', the 
> error info shows me that it can't connect to zookeeper server(timeout).
>     However, if I change back to 0.5.0 version, everything works well. 
> Did I miss something or it's  an issue?
> Thanks,
> Larry
>
> On 03/13/2013 01:29 AM, Matthieu Morel wrote:
>> (For some reason my answer sent through apache's stmp does not seem 
>> to reach destination. So here it is again - ignore the other mail if 
>> it finally gets delivered)
>>
>>
>> Hi Larry,
>>
>> If you are totally new to S4, you are right on time to use the latest 
>> version. We are currently voting on a first release candidate for S4 
>> 0.6.0, which is available there: 
>> http://people.apache.org/~mmorel/s4-0.6.0-incubating-release-candidate-1/ 
>> <http://people.apache.org/%7Emmorel/s4-0.6.0-incubating-release-candidate-1/>
>> with updated documentation (still a few updates to add) here: 
>> http://incubator.apache.org/s4/doc/0.6.0
>>
>> I'll answer your questions inline and refer to the 0.6 release when 
>> appropriate.
>>
>>
>> On Mar 12, 2013, at 11:49 , larryzhang wrote:
>>
>>> Hi,
>>>    I'm totally new to S4, and want to run the counter example in 
>>> 'apache-s4-...-source/subprojects/s4-examples/org/apache/s4/example/counter',

>>> I have compiled all codes using gradle, and also type 'gradle 
>>> eclipse' under the project root folder 'apache-s4-...source/'. Then 
>>> I import all subprojects to eclipse.
>>>    I found most of the codes in this example is easy to understand, 
>>> except 'Module.java', I didn't know the following things:
>>>      a) in function 'loadProperties':
>>>            InputStream is = 
>>> this.getClass().getResourceAsStream("/s4-piper-example.properties");
>>> *Q1: where is the file 's4-piper-example.properties', and what does 
>>> it used for?* I searched web and find these contents:
>>>                       --------------
>>>                       pe.counter.interval = 1
>>>                       comm.queue_emmiter_size = 8000
>>>                       comm.queue_listener_size = 8000
>>>                       cluster.hosts = localhost
>>>                       cluster.ports = 5077
>>>                       cluster.lock_dir = /tmp
>>>                      ---------------
>>> *Is that main the cluster environment must be set up firstly?*
>>
>> The example is not maintained, even for 0.5.0, see 
>> https://github.com/apache/incubator-s4/blob/0.5.0/subprojects/s4-example/src/main/java/org/apache/s4/example/counter/README.md
>>
>>>
>>>      b) In function 'configure', all codes are commented, e.g:
>>>             ----------------
>>>              // bind(MyApp.class);
>>>             //
>>>             // bind(PhysicalCluster.class);
>>>            //
>>>            // /* Configure static assignment using a configuration 
>>> file. */
>>>            // bind(Assignment.class).to(AssignmentFromFile.class);
>>>            ------------------
>>> *Q2: shall I need to change something in this function?* I note that 
>>> 'Emitter and Listener' need to be bind, I thought these are like 
>>> 'adapter' which I saw in the 
>>> walkthroughhttps://cwiki.apache.org/confluence/display/S4/S4+piper+walkthrough,

>>> which are used to define source, am I right?
>>
>> You may ignore this commented code. We are currently adding 
>> clarifications on the ways to inject events into S4 0.6 in the 
>> website documentation (i.e. through adapters or directly from the App 
>> class)
>>
>>>
>>>       c) I note MyApp.java has main class, so I run it directly in 
>>> eclipse, and then got lots of errors occurs:
>>>
>>> ---------------------------------------------------------------------------------------------
>>> exception in thread "main" com.google.inject.ConfigurationException: 
>>> Guice configuration errors:
>>> 1) No implementation for java.lang.Integer annotated with 
>>> @com.google.inject.name.Named(value=s4.cluster.zk_connection_timeout) was 
>>> bound.
>>>   while locating java.lang.Integer annotated with 
>>> @com.google.inject.name.Named(value=s4.cluster.zk_connection_timeout)
>>>     for parameter 2 at 
>>> org.apache.s4.comm.topology.RemoteStreams.<init>(RemoteStreams.java:92)
>>>   while locating org.apache.s4.comm.topology.RemoteStreams
>>>     for field at 
>>> org.apache.s4.core.RemoteSenders.streams(RemoteSenders.java:43)
>>>   while locating org.apache.s4.core.RemoteSenders
>>>     for field at org.apache.s4.core.App.remoteSenders(App.java:47)
>>>   while locating org.apache.s4.example.counter.MyApp
>>>
>>> 2) No implementation for java.lang.Integer annotated with 
>>> @com.google.inject.name.Named(value=s4.cluster.zk_connection_timeout) was 
>>> bound.
>>>   while locating java.lang.Integer annotated with 
>>> @com.google.inject.name.Named(value=s4.cluster.zk_connection_timeout)
>>>     for parameter 2 at 
>>> org.apache.s4.comm.topology.RemoteStreams.<init>(RemoteStreams.java:92)
>>>   while locating org.apache.s4.comm.topology.RemoteStreams
>>>     for field at org.apache.s4.core.App.remoteStreams(App.java:47)
>>>   while locating org.apache.s4.example.counter.MyApp
>>>
>>>    ....  ....
>>> 7) No implementation for org.apache.s4.base.Emitter was bound.
>>>   while locating org.apache.s4.base.Emitter
>>>     for parameter 0 at org.apache.s4.core.Sender.<init>(Sender.java:62)
>>>   while locating org.apache.s4.core.Sender
>>>     for field at org.apache.s4.core.App.sender(App.java:47)
>>>   while locating org.apache.s4.example.counter.MyApp
>>> 8) No implementation for org.apache.s4.base.Hasher was bound.
>>>   while locating org.apache.s4.base.Hasher
>>>     for parameter 2 at org.apache.s4.core.Sender.<init>(Sender.java:62)
>>>   while locating org.apache.s4.core.Sender
>>>     for field at org.apache.s4.core.App.sender(App.java:47)
>>>   while locating org.apache.s4.example.counter.MyApp
>>>    ...   ....
>>> ------------------------------------------------------------------------------------------------------------------------
>>> *Q3: I know these errors are related to my above questions, how can 
>>> I handle these?*
>>
>> You cannot run an S4 app independently of S4. See 
>> http://incubator.apache.org/s4/doc/0.6.0/dev_tips/ for how to do 
>> that. Basically you need to upload the application configuration to 
>> the cluster manager first, then start the node.
>>
>>> *Q4: One more thing, in main function, a Sender and a Receiver are 
>>> created,* which confuse me because I didn't see these in walkthrough 
>>> page, *what does these mean?*
>>>           -----------------------------
>>>        Sender sender = injector.getInstance(Sender.class);
>>>         Receiver receiver = injector.getInstance(Receiver.class);
>>>          -----------------------------
>>>
>>
>> Again, you are referring the the counter app which is not maintained. 
>> I recommend you look at the walkthrough and the twitter app for a 
>> more complete example.
>>
>>
>> Hope this helps,
>>
>> Matthieu
>>
>> On Mar 12, 2013, at 11:49 , larryzhang wrote:
>>
>>> Hi,
>>>    I'm totally new to S4, and want to run the counter example in 
>>> 'apache-s4-...-source/subprojects/s4-examples/org/apache/s4/example/counter',

>>> I have compiled all codes using gradle, and also type 'gradle 
>>> eclipse' under the project root folder 'apache-s4-...source/'. Then 
>>> I import all subprojects to eclipse.
>>>    I found most of the codes in this example is easy to understand, 
>>> except 'Module.java', I didn't know the following things:
>>>      a) in function 'loadProperties':
>>>            InputStream is = 
>>> this.getClass().getResourceAsStream("/s4-piper-example.properties");
>>> *Q1: where is the file 's4-piper-example.properties', and what does 
>>> it used for?* I searched web and find these contents:
>>>                       --------------
>>>                       pe.counter.interval = 1
>>>                       comm.queue_emmiter_size = 8000
>>>                       comm.queue_listener_size = 8000
>>>                       cluster.hosts = localhost
>>>                       cluster.ports = 5077
>>>                       cluster.lock_dir = /tmp
>>>                      ---------------
>>> *Is that main the cluster environment must be set up firstly?*
>>>
>>>      b) In function 'configure', all codes are commented, e.g:
>>>             ----------------
>>>              // bind(MyApp.class);
>>>             //
>>>             // bind(PhysicalCluster.class);
>>>            //
>>>            // /* Configure static assignment using a configuration 
>>> file. */
>>>            // bind(Assignment.class).to(AssignmentFromFile.class);
>>>            ------------------
>>> *Q2: shall I need to change something in this function?* I note that 
>>> 'Emitter and Listener' need to be bind, I thought these are like 
>>> 'adapter' which I saw in the walkthrough 
>>> https://cwiki.apache.org/confluence/display/S4/S4+piper+walkthrough, 
>>> which are used to define source, am I right?
>>>
>>>       c) I note MyApp.java has main class, so I run it directly in 
>>> eclipse, and then got lots of errors occurs:
>>>
>>> ---------------------------------------------------------------------------------------------
>>> exception in thread "main" com.google.inject.ConfigurationException: 
>>> Guice configuration errors:
>>> 1) No implementation for java.lang.Integer annotated with 
>>> @com.google.inject.name.Named(value=s4.cluster.zk_connection_timeout) was 
>>> bound.
>>>   while locating java.lang.Integer annotated with 
>>> @com.google.inject.name.Named(value=s4.cluster.zk_connection_timeout)
>>>     for parameter 2 at 
>>> org.apache.s4.comm.topology.RemoteStreams.<init>(RemoteStreams.java:92)
>>>   while locating org.apache.s4.comm.topology.RemoteStreams
>>>     for field at 
>>> org.apache.s4.core.RemoteSenders.streams(RemoteSenders.java:43)
>>>   while locating org.apache.s4.core.RemoteSenders
>>>     for field at org.apache.s4.core.App.remoteSenders(App.java:47)
>>>   while locating org.apache.s4.example.counter.MyApp
>>>
>>> 2) No implementation for java.lang.Integer annotated with 
>>> @com.google.inject.name.Named(value=s4.cluster.zk_connection_timeout) was 
>>> bound.
>>>   while locating java.lang.Integer annotated with 
>>> @com.google.inject.name.Named(value=s4.cluster.zk_connection_timeout)
>>>     for parameter 2 at 
>>> org.apache.s4.comm.topology.RemoteStreams.<init>(RemoteStreams.java:92)
>>>   while locating org.apache.s4.comm.topology.RemoteStreams
>>>     for field at org.apache.s4.core.App.remoteStreams(App.java:47)
>>>   while locating org.apache.s4.example.counter.MyApp
>>>
>>>    ....  ....
>>> 7) No implementation for org.apache.s4.base.Emitter was bound.
>>>   while locating org.apache.s4.base.Emitter
>>>     for parameter 0 at org.apache.s4.core.Sender.<init>(Sender.java:62)
>>>   while locating org.apache.s4.core.Sender
>>>     for field at org.apache.s4.core.App.sender(App.java:47)
>>>   while locating org.apache.s4.example.counter.MyApp
>>> 8) No implementation for org.apache.s4.base.Hasher was bound.
>>>   while locating org.apache.s4.base.Hasher
>>>     for parameter 2 at org.apache.s4.core.Sender.<init>(Sender.java:62)
>>>   while locating org.apache.s4.core.Sender
>>>     for field at org.apache.s4.core.App.sender(App.java:47)
>>>   while locating org.apache.s4.example.counter.MyApp
>>>    ...   ....
>>> ------------------------------------------------------------------------------------------------------------------------
>>> *Q3: I know these errors are related to my above questions, how can 
>>> I handle these?*
>>> *Q4: One more thing, in main function, a Sender and a Receiver are 
>>> created,* which confuse me because I didn't see these in walkthrough 
>>> page, *what does these mean?*
>>>           -----------------------------
>>>        Sender sender = injector.getInstance(Sender.class);
>>>         Receiver receiver = injector.getInstance(Receiver.class);
>>>          -----------------------------
>>>
>>>    I'll appreciate any help you give, thanks a lot!
>>> Regards,
>>> larry
>>>
>>
>


Mime
View raw message