calcite-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Julian Hyde <jh...@apache.org>
Subject Re: Piglet update
Date Mon, 13 Feb 2017 20:38:03 GMT
I’ve not worked on Piglet after the initial spike. But the point of that work was to create
something that someone else could pick up, as you have done.

If the Grunt parser is working better for you, we could consider using that instead of my
hand-rolled parser. (But hopefully we can keep the VALUES operator; it’s a shame Pig latin
doesn’t have this.)

Yes, there are a number of issues outstanding. There is actually a test case for nested FOREACH
in PigletTest.java, disabled. I figured we can work on these when people start using Piglet
for real work. Log JIRA cases for the missing features and we can discuss how they could be
implemented.

Are you aware of the work Eli Levine is doing on the Pig Adapter[1]? (It’s the opposite
of Piglet — Pig on the bottom, rather than on the top — but it proves there is interest
relating to Calcite-Pig integration.)

Julian

[1] https://issues.apache.org/jira/browse/CALCITE-1598 <https://issues.apache.org/jira/browse/CALCITE-1598>



> On Feb 13, 2017, at 12:16 PM, Khai Tran <khtran@linkedin.com.INVALID> wrote:
> 
> Hi all,
> 
> I just want to check if anyone is working on Piglet. I've not seen any
> commits on the source for awhile.
> 
> I had ~3K LOC for converting Pig scripts into Calcite logical plans. My
> approach is different from the one in Piglet. I used Grunt Parser from Pig
> to parse Pig scripts and then write the code to convert Pig logical plans
> into Calcite logical plans. So technically, we can translating any Pig
> scripts into Calcite plans (and then into SQL).
> 
> The interesting points about the conversion are about handling Pig UDFs,
> Pig flexible schemas. and converting Pig nested foreach operators
> 
> The code will be tested in LinkedIn production. Just want to check if there
> are interests in the community so that I can ask for permission to public
> the code.
> 
> Thanks,
> Khai


Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message