gora-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Renato MarroquĂ­n Mogrovejo <renatoj.marroq...@gmail.com>
Subject Re: GSoC Ideas
Date Wed, 14 Mar 2018 22:22:39 GMT
Hey guys,

There might not be an integration/convertors of Arrow to Avro (and/or
viceversa) because there are parquet readers that can take avro and once
stuff is in parquet, then arrow can be used directly.
Regarding if an integration of Parquet with Gora, I think it would be
interesting to make it easier for people to read and write parquet files by
providing a higher level api as Gora provides. However, for you @Talat,
that knows Gora pretty well, maybe you could take another project that
helps Gora more. For example, fixing the integration with Nutch. There are
multiple loose ends in Nutch 2.x and Gora that we have neglected as a
community.
IMHO that should be GSOC project. I can't mentored it because I do not have
enough insights on this, but @Lewis and @Talat you can probably tackle this
as mentor and student. This would be an awesome contribution to the project
as there are quite a lot of people going over Nutch and trying to use it
with Gora.
Just my 2c


Renato M.

2018-03-13 17:23 GMT+01:00 lewis john mcgibbney <lewismc@apache.org>:

> Hi Talat,
>
> On Tue, Mar 13, 2018 at 9:07 AM, Talat Uyarer <talat@uyarer.com> wrote:
>
>> Hi Lewis,
>> Yes I checked the  Jira i saw Redis and Apache Ignite integration. I just
>> wanted to ask you guys how more curial issues.
>>
>> What about Apache Arrow ? Most of projects start using that. I have one
>> concern about it. Parquet and Arrow both of them are columnar. Avro provide
>> us flexibility.
>> [1] https://arrow.apache.org/
>>
>>
>> I agree. There is certainly some thought which needs to go into
> integration of these formats. Note, that Arrow does not have an Avro
> integration yet, so that would be indicative that the data modeling is non
> trivial or else I suspect it would have already been done.
> @Renato, you have some thoughts on this?
> Lewis
>

Mime
View raw message