gora-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Lahiru Jayasekera <mlpjayasek...@gmail.com>
Subject Re: POC for hazelcast jet integration
Date Fri, 26 Apr 2019 13:02:05 GMT
Hi Madhawa,
Sorry for the late reply. Sure I'll try that and let you know.

Thanks

On Tue, Apr 23, 2019 at 2:24 PM Madhawa Kasun Gunasekara <
madhawa30@gmail.com> wrote:

> Hi Lahiru,
>
> Good initiative.
> It seems like we need to use hadoop-hdfs and hadoop common with version
> 2.8.3. Try adding these dependencies, and exclude the old dependency from
> the gora, or otherwise, we can do a version upgrade in gora for Hadoop. at
> the moment we use Hadoop version 2.5.2 in gora but I prefer to upgrade the
> Hadoop versions in the gora to 2.8.3.
>
> Recently hazelcast developers released 3.0 version, We can try that also.
>
> Thanks,
> Madhawa
>
>
> On Sat, Apr 6, 2019 at 9:53 PM Lahiru Jayasekera <mlpjayasekera@gmail.com>
> wrote:
>
> > Hi all,
> > I have come up with a poc for hazelcast jet execution engine support for
> > gora.
> >
> >
> https://github.com/LahiruJayasekara/gora/blob/poc-hazelcast-jet/gora-tutorial/src/main/java/org/apache/gora/tutorial/log/HazelcastJetPOC.java
> >
> > Here I have wrote a custom source for jet. What this does is read the
> > AccessLog table created via LogManager example, and feed the PageView
> > objects to jet.
> >
> > I have also looked into using hdfs connectors of jet. But I encountered a
> > problem with conflicting hadoop versions. InputFormat class used in gora
> is
> > from pacjage 'org.apache.hadoop.mapreduce', but in jet it is '
> > org.apache.hadoop.mapred.InputFormat'.
> >
> > Anyway the proposed poc works as expected. Please add your suggestions
> > here.
> >
> > Thanks and regards
> >
> > --
> > Lahiru Jayasekara
> > Batch 15
> > Faculty of Information Technology
> > University of Moratuwa
> > 0716492170
> >
>


-- 
Lahiru Jayasekara
Batch 15
Faculty of Information Technology
University of Moratuwa
0716492170

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message