incubator-s4-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Matthieu Morel <matth...@yahoo-inc.com>
Subject Re: How can I run counter example in s4 piper
Date Tue, 12 Mar 2013 17:29:38 GMT
(For some reason my answer sent through apache's stmp does not seem to reach destination. So
here it is again - ignore the other mail if it finally gets delivered)


Hi Larry,

If you are totally new to S4, you are right on time to use the latest version. We are currently
voting on a first release candidate for S4 0.6.0, which is available there: http://people.apache.org/~mmorel/s4-0.6.0-incubating-release-candidate-1/
with updated documentation (still a few updates to add) here: http://incubator.apache.org/s4/doc/0.6.0

I'll answer your questions inline and refer to the 0.6 release when appropriate.


On Mar 12, 2013, at 11:49 , larryzhang wrote:

Hi,
   I'm totally new to S4, and want to run the counter example in 'apache-s4-...-source/subprojects/s4-examples/org/apache/s4/example/counter',
I have compiled all codes using gradle, and also type 'gradle eclipse' under the project root
folder 'apache-s4-...source/'. Then I import all subprojects to eclipse.
   I found most of the codes in this example is easy to understand, except 'Module.java',
I didn't know the following things:
     a) in function 'loadProperties':
           InputStream is = this.getClass().getResourceAsStream("/s4-piper-example.properties");
             Q1: where is the file 's4-piper-example.properties', and what does it used for?
I searched web and find these contents:
                      --------------
                      pe.counter.interval = 1
                      comm.queue_emmiter_size = 8000
                      comm.queue_listener_size = 8000
                      cluster.hosts = localhost
                      cluster.ports = 5077
                      cluster.lock_dir = /tmp
                     ---------------
                Is that main the cluster environment must be set up firstly?

The example is not maintained, even for 0.5.0, see https://github.com/apache/incubator-s4/blob/0.5.0/subprojects/s4-example/src/main/java/org/apache/s4/example/counter/README.md


     b) In function 'configure', all codes are commented, e.g:
            ----------------
             // bind(MyApp.class);
            //
            // bind(PhysicalCluster.class);
           //
           // /* Configure static assignment using a configuration file. */
           // bind(Assignment.class).to(AssignmentFromFile.class);
           ------------------
           Q2: shall I need to change something in this function? I note that 'Emitter and
Listener' need to be bind, I thought these are like 'adapter' which I saw in the walkthroughhttps://cwiki.apache.org/confluence/display/S4/S4+piper+walkthrough,
which are used to define source, am I right?

You may ignore this commented code. We are currently adding clarifications on the ways to
inject events into S4 0.6 in the website documentation (i.e. through adapters or directly
from the App class)


      c) I note MyApp.java has main class, so I run it directly in eclipse, and then got lots
of errors occurs:

     ---------------------------------------------------------------------------------------------
exception in thread "main" com.google.inject.ConfigurationException: Guice configuration errors:
1) No implementation for java.lang.Integer annotated with @com.google.inject.name.Named(value=s4.cluster.zk_connection_timeout)
was bound.
  while locating java.lang.Integer annotated with @com.google.inject.name.Named(value=s4.cluster.zk_connection_timeout)
    for parameter 2 at org.apache.s4.comm.topology.RemoteStreams.<init>(RemoteStreams.java:92)
  while locating org.apache.s4.comm.topology.RemoteStreams
    for field at org.apache.s4.core.RemoteSenders.streams(RemoteSenders.java:43)
  while locating org.apache.s4.core.RemoteSenders
    for field at org.apache.s4.core.App.remoteSenders(App.java:47)
  while locating org.apache.s4.example.counter.MyApp

2) No implementation for java.lang.Integer annotated with @com.google.inject.name.Named(value=s4.cluster.zk_connection_timeout)
was bound.
  while locating java.lang.Integer annotated with @com.google.inject.name.Named(value=s4.cluster.zk_connection_timeout)
    for parameter 2 at org.apache.s4.comm.topology.RemoteStreams.<init>(RemoteStreams.java:92)
  while locating org.apache.s4.comm.topology.RemoteStreams
    for field at org.apache.s4.core.App.remoteStreams(App.java:47)
  while locating org.apache.s4.example.counter.MyApp

   ....  ....
7) No implementation for org.apache.s4.base.Emitter was bound.
  while locating org.apache.s4.base.Emitter
    for parameter 0 at org.apache.s4.core.Sender.<init>(Sender.java:62)
  while locating org.apache.s4.core.Sender
    for field at org.apache.s4.core.App.sender(App.java:47)
  while locating org.apache.s4.example.counter.MyApp
8) No implementation for org.apache.s4.base.Hasher was bound.
  while locating org.apache.s4.base.Hasher
    for parameter 2 at org.apache.s4.core.Sender.<init>(Sender.java:62)
  while locating org.apache.s4.core.Sender
    for field at org.apache.s4.core.App.sender(App.java:47)
  while locating org.apache.s4.example.counter.MyApp
   ...   ....
------------------------------------------------------------------------------------------------------------------------
    Q3: I know these errors are related to my above questions, how can I handle these?

You cannot run an S4 app independently of S4. See http://incubator.apache.org/s4/doc/0.6.0/dev_tips/
for how to do that. Basically you need to upload the application configuration to the cluster
manager first, then start the node.

    Q4: One more thing, in main function, a Sender and a Receiver are created, which confuse
me because I didn't see these in walkthrough page, what does these mean?
          -----------------------------
       Sender sender = injector.getInstance(Sender.class);
        Receiver receiver = injector.getInstance(Receiver.class);
         -----------------------------


Again, you are referring the the counter app which is not maintained. I recommend you look
at the walkthrough and the twitter app for a more complete example.


Hope this helps,

Matthieu

On Mar 12, 2013, at 11:49 , larryzhang wrote:

Hi,
   I'm totally new to S4, and want to run the counter example in 'apache-s4-...-source/subprojects/s4-examples/org/apache/s4/example/counter',
I have compiled all codes using gradle, and also type 'gradle eclipse' under the project root
folder 'apache-s4-...source/'. Then I import all subprojects to eclipse.
   I found most of the codes in this example is easy to understand, except 'Module.java',
I didn't know the following things:
     a) in function 'loadProperties':
           InputStream is = this.getClass().getResourceAsStream("/s4-piper-example.properties");
             Q1: where is the file 's4-piper-example.properties', and what does it used for?
I searched web and find these contents:
                      --------------
                      pe.counter.interval = 1
                      comm.queue_emmiter_size = 8000
                      comm.queue_listener_size = 8000
                      cluster.hosts = localhost
                      cluster.ports = 5077
                      cluster.lock_dir = /tmp
                     ---------------
                Is that main the cluster environment must be set up firstly?

     b) In function 'configure', all codes are commented, e.g:
            ----------------
             // bind(MyApp.class);
            //
            // bind(PhysicalCluster.class);
           //
           // /* Configure static assignment using a configuration file. */
           // bind(Assignment.class).to(AssignmentFromFile.class);
           ------------------
           Q2: shall I need to change something in this function? I note that 'Emitter and
Listener' need to be bind, I thought these are like 'adapter' which I saw in the walkthrough
https://cwiki.apache.org/confluence/display/S4/S4+piper+walkthrough, which are used to define
source, am I right?

      c) I note MyApp.java has main class, so I run it directly in eclipse, and then got lots
of errors occurs:

     ---------------------------------------------------------------------------------------------
exception in thread "main" com.google.inject.ConfigurationException: Guice configuration errors:
1) No implementation for java.lang.Integer annotated with @com.google.inject.name.Named(value=s4.cluster.zk_connection_timeout)
was bound.
  while locating java.lang.Integer annotated with @com.google.inject.name.Named(value=s4.cluster.zk_connection_timeout)
    for parameter 2 at org.apache.s4.comm.topology.RemoteStreams.<init>(RemoteStreams.java:92)
  while locating org.apache.s4.comm.topology.RemoteStreams
    for field at org.apache.s4.core.RemoteSenders.streams(RemoteSenders.java:43)
  while locating org.apache.s4.core.RemoteSenders
    for field at org.apache.s4.core.App.remoteSenders(App.java:47)
  while locating org.apache.s4.example.counter.MyApp

2) No implementation for java.lang.Integer annotated with @com.google.inject.name.Named(value=s4.cluster.zk_connection_timeout)
was bound.
  while locating java.lang.Integer annotated with @com.google.inject.name.Named(value=s4.cluster.zk_connection_timeout)
    for parameter 2 at org.apache.s4.comm.topology.RemoteStreams.<init>(RemoteStreams.java:92)
  while locating org.apache.s4.comm.topology.RemoteStreams
    for field at org.apache.s4.core.App.remoteStreams(App.java:47)
  while locating org.apache.s4.example.counter.MyApp

   ....  ....
7) No implementation for org.apache.s4.base.Emitter was bound.
  while locating org.apache.s4.base.Emitter
    for parameter 0 at org.apache.s4.core.Sender.<init>(Sender.java:62)
  while locating org.apache.s4.core.Sender
    for field at org.apache.s4.core.App.sender(App.java:47)
  while locating org.apache.s4.example.counter.MyApp
8) No implementation for org.apache.s4.base.Hasher was bound.
  while locating org.apache.s4.base.Hasher
    for parameter 2 at org.apache.s4.core.Sender.<init>(Sender.java:62)
  while locating org.apache.s4.core.Sender
    for field at org.apache.s4.core.App.sender(App.java:47)
  while locating org.apache.s4.example.counter.MyApp
   ...   ....
------------------------------------------------------------------------------------------------------------------------
    Q3: I know these errors are related to my above questions, how can I handle these?
    Q4: One more thing, in main function, a Sender and a Receiver are created, which confuse
me because I didn't see these in walkthrough page, what does these mean?
          -----------------------------
       Sender sender = injector.getInstance(Sender.class);
        Receiver receiver = injector.getInstance(Receiver.class);
         -----------------------------

   I'll appreciate any help you give, thanks a lot!
Regards,
larry



Mime
View raw message