gora-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From GitBox <...@apache.org>
Subject [GitHub] [gora] LahiruJayasekara opened a new pull request #175: GORA-546 Hazelcast Jet execution engine support
Date Tue, 16 Jul 2019 15:25:59 GMT
LahiruJayasekara opened a new pull request #175: GORA-546 Hazelcast Jet execution engine support
URL: https://github.com/apache/gora/pull/175
 
 
   Don't merge
   Please review
   Your feed back is mostly welcome
   
   In this PR I have implemented the new jet source and sink connectors for Gora.
   I have added some test cases to check the functionality of those connectors.
   
   Following is an example query for word counting using the new connectors.
   
   ```
   JetEngine<Long, Pageview, Long, ResultPageView> jetEngine = new JetEngine<>();
       BatchSource<JetInputOutputFormat<Long, Pageview>> fileSource = jetEngine.createDataSource(dataStore,
query);
       Pipeline p = Pipeline.create();
       p.drawFrom(fileSource)
           .filter(item -> item.getValue().getIp().toString().equals("88.240.129.183"))
           .map(e -> {
             ResultPageView resultPageView = new ResultPageView();
             resultPageView.setIp(e.getValue().getIp());
             resultPageView.setTimestamp(e.getValue().getTimestamp());
             resultPageView.setUrl(e.getValue().getUrl());
             return new JetInputOutputFormat<Long, ResultPageView>(e.getValue().getTimestamp(),
resultPageView);
           })
           .drainTo(jetEngine.createDataSink(dataStoreOut));
   ```

----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
users@infra.apache.org


With regards,
Apache Git Services

Mime
View raw message