incubator-s4-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From 杨定裕 <yangdin...@gmail.com>
Subject Re: How to use S4 to implement Join operator
Date Thu, 04 Oct 2012 04:42:17 GMT
Hi, I am running at the testCluster with two adapter.
The question is that the list stores all the input data. In fact, it should
store part data in each PE.
The list in each PE should be independent, not store all the data.
How can i implement this?
*
This is the reslut:

OLAJoinPE------:Table:!||| T_lineitem.size(): 1||| PEID:1||| count:1
OLAJoinPE------:Table:!||| T_lineitem.size(): 2||| PEID:1||| count:2
OLAJoinPE------:Table:!||| T_lineitem.size(): 3||| PEID:1||| count:3
OLAJoinPE------:Table:!||| T_lineitem.size(): 4||| PEID:1||| count:4
OLAJoinPE------:Table:!||| T_lineitem.size(): 5||| PEID:1||| count:5
OLAJoinPE------:Table:!||| T_lineitem.size(): 6||| PEID:1||| count:6
OLAJoinPE------:Table:!||| T_lineitem.size(): 7||| PEID:2||| count:1
OLAJoinPE------:Table:!||| T_lineitem.size(): 8||| PEID:3||| count:1
OLAJoinPE------:Table:!||| T_lineitem.size(): 9||| PEID:3||| count:2
OLAJoinPE------:Table:!||| T_lineitem.size(): 10||| PEID:3||| count:3
OLAJoinPE------:Table:!||| T_lineitem.size(): 11||| PEID:3||| count:4
OLAJoinPE------:Table:!||| T_lineitem.size(): 12||| PEID:3||| count:5
OLAJoinPE------:Table:!||| T_lineitem.size(): 13||| PEID:3||| count:6
OLAJoinPE------:Table:!||| T_lineitem.size(): 14||| PEID:4||| count:1
OLAJoinPE------:Table:!||| T_lineitem.size(): 15||| PEID:5||| count:1
OLAJoinPE------:Table:!||| T_lineitem.size(): 16||| PEID:5||| count:2
OLAJoinPE------:Table:!||| T_lineitem.size(): 17||| PEID:5||| count:3
OLAJoinPE------:Table:!||| T_lineitem.size(): 18||| PEID:6||| count:1
OLAJoinPE------:Table:!||| T_lineitem.size(): 19||| PEID:7||| count:1
OLAJoinPE------:Table:!||| T_lineitem.size(): 20||| PEID:7||| count:2



*
2012/10/4 Shailendra Mishra <shailendrah@gmail.com>

> Ok, this looks correct, so then what is the question. Do you have this
> app running in more than one partition ? - Shailendra
>
> On Wed, Oct 3, 2012 at 9:18 PM, 杨定裕 <yangdingyu@gmail.com> wrote:
> > Hi, Shailendra,
> > Yes, I have a stream with two event types:lineitem and order. The same
> key
> > will be sent to same PE.
> >
> > this is the code of mine:
> > ---------------------------
> >
> >     private List<Event> T_lineitem = new ArrayList<Event>();
> >     private List<Event> T_order = new ArrayList<Event>();
> >
> >     private long count = 0;
> >
> >     public void onEvent(Event event) {
> >         // in this example, we use the default generic Event type, by you
> > can also define your own type
> >         String value = "";
> >         count = count+1;
> >         try{
> >             value = event.get("lineitem").replace("|","&").split("&")[0];
> >
> >             T_lineitem.add(event);
> >             System.out.println("JoinPE------:Table:!|||
> T_lineitem.size():
> > "+T_lineitem.size()+"||| PEID:"+getId()+"||| count:"+count);
> >         }catch(Exception e){
> >
> >         }
> >         try{
> >             value = event.get("order").replace("|","&").split("&")[0];
> >             T_order.add(event);
> >             System.out.println("oinPE------:Table:!||| T_order.size():
> > "+T_order.size()+"||| PEID:"+getId()+"||| count:"+count);
> >
> >
> >         }catch(Exception e){
> >
> >         }
> > }
> > -----------------------------
> >
> > The output is like this :
> > JoinPE------:Table:!||| T_lineitem.size(): 19||| PEID:7||| count:1
> > JoinPE------:Table:!||| T_lineitem.size(): 20||| PEID:7||| count:2
> >
> > That is the problem, count should be more than the size of list.
> >
> > Dingyu
> >
>

Mime
View raw message