drill-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Charles Givre <cgi...@gmail.com>
Subject Re: Working on upgrade Drill Calcite version
Date Fri, 19 Jan 2018 13:30:56 GMT
Great work!

> On Jan 19, 2018, at 05:43, Arina Ielchiieva <arina@apache.org> wrote:
> 
> I am happy to announce that Apache Drill has rebased on Calcite 1.15. All
> information can be found in DRILL-3993 and in Google Doc [1].
> Currently changes are in master and are intended to be part of Drill 1.13.0
> release unless something critical pops up. Feel free to test them and give
> your feedback.
> 
> Thanks Roman Kulyk and Volodymyr Vysotskyi for making this happen.
> Aman Sinha and Chunhui Shi for the code review.
> And of course Julian Hyde for helping to address / review Calcite specific
> issues.
> 
> [1]
> https://docs.google.com/document/d/1Lqk9NoKQviz0YimBmov4z1pui7QjJGjDVwMa1p0emPk/edit#heading=h.i3rowg20vxv4
> 
> 
> Kind regards
> Arina
> 
> On Fri, Sep 1, 2017 at 8:01 PM, Roman Kulyk <rom.kulyk@gmail.com> wrote:
> 
>> Team, thanks for your answers!
>> 
>> Julian, сontinuing the topic of Calcite tests I just want to note that I
>> have ignored 7 test in 5ba68240 commit in Calcite. Here is a list of
>> ignored tests:
>> 
>>  TraitConversionTest.testTraitConversion
>>  JdbcAdapterTest.testCrossJoinWithJoinKeyPlan
>>  JdbcAdapterTest.testCrossJoinWithJoinKeyAndFilterPlan
>>  JdbcTest.testJoinManyWay (it was also ignored by 8a87ad1e5)
>>  LatticeTest.testTileAlgorithm (it was also ignored by 2e076e1a)
>>  LatticeTest.testTileAlgorithm2 (it was also ignored by 8a87ad1e5)
>>  LatticeTest.testTwoLattices
>> 
>> You can see detailed information in my comments. Also, I want to notice
>> that there are 2 required commits in my Calcite branch: 5ba68240 and
>> 1f5145b1. Without them, we lose a big part of Drill functionality.
>> 
>> Best regards,
>> Roman Kulyk
>> 
>> 2017-08-31 18:43 GMT+00:00 Julian Hyde <jhyde@apache.org>:
>> 
>>> +dev@calcite.
>>> 
>>> Excellent work, Roman.
>>> 
>>> Your changes to Calcite don’t seem to be very major, and I ran the test
>>> suite on your https://github.com/KulykRoman/incubator-calcite/commits/
>>> DrillCalcite1.13.0_rc1 branch and all the tests pass. So when you are
>>> ready, I think we can expedite getting these changes back into Calcite.
>>> 
>>> Calcite will release 1.14 probably in the next two weeks. So, when you’re
>>> on to 1.13 let’s continue the momentum and get up to the latest release.
>> I
>>> would love to get to the situation that we can ask Drill to verify a
>>> Calcite release before we make it. There are possibly breaking changes in
>>> the area of schema and grouping-sets aggregate but it should be mostly
>>> straightforward.
>>> 
>>> Julian
>>> 
>>> 
>>>> On Aug 31, 2017, at 8:12 AM, Jinfeng Ni <jni@apache.org> wrote:
>>>> 
>>>> It's great to hear that someone is working on moving Drill on new
>> Calcite
>>>> release! As someone who did similar job in last two rebase work, I
>> could
>>>> see that there would be many regressions to fix. Please let me know if
>> I
>>>> can provide any help.
>>>> 
>>>> I thought we have pushed almost Drill specific commits to Calcite. I'll
>>>> take a look at your Calcite branch. Ideally, we want to get rid of all
>>>> Drill specific commits, after the rebase work this time.
>>>> 
>>>> Thanks,
>>>> 
>>>> Jinfeng
>>>> 
>>>> 
>>>> 
>>>> 
>>>> On Thu, Aug 31, 2017 at 8:04 AM, Aman Sinha <amansinha@apache.org>
>>> wrote:
>>>> 
>>>>> Yes, this is long overdue !  Thanks for working on it Roman.   If
>>> needed,
>>>>> we can do a separate hangout with a few selected folks who have worked
>>> on
>>>>> Drill+Calcite to provide feedback on how to resolve the test failures.
>>>>> Although, at this stage you probably know a lot more about the
>>> integration
>>>>> than many of us.
>>>>> 
>>>>> -Aman
>>>>> 
>>>>> On Thu, Aug 31, 2017 at 7:33 AM, Roman Kulyk <rom.kulyk@gmail.com>
>>> wrote:
>>>>> 
>>>>>> Hi, team!
>>>>>> 
>>>>>> I want to inform that I am working on updating Drill Calcite version
>> to
>>>>>> 1.13. I forked from Calcite "branch-1.13" and added some commits:
>>>>>> - DRILL-1455: Add return type-inference strategy for arithmetic
>>> operators
>>>>>> when one of the arguments is ANY type. (was not in Calcite)
>>>>>> - Add new method to ViewExpander interface to allow passing
>> SchemaRoot.
>>>>>> (was not in Calcite)
>>>>>> - Allow a MAP literal type. (was not in Calcite)
>>>>>> - DRILL-4047: Modify table functions to allow querying a table with
>>>>> options
>>>>>> in Drill (was not in Calcite)
>>>>>> - Drill-specific change: Add back AbstractConverter in RelSet.java
>> ...
>>>>>> (small changes after a3bc0d8 commit)
>>>>>> - CALCITE-628 related but not fix the problem: Ensure target traits
>> are
>>>>>> simple when use Frameworks or RelOptRule.convert() method. (small
>>> changes
>>>>>> after a3bc0d8 commit)
>>>>>> -  Support select * from schema-less table in execution engine like
>>> Drill
>>>>>> (small changes after 9bd7d75 commit)
>>>>>> 
>>>>>> Only after adding these commits to Calcite, a part of  Drill
>>>>> functionality
>>>>>> started to work as before. Currently, I am at Drill integration
>> stage.
>>> I
>>>>> am
>>>>>> fixing Drill unit tests. Progress can be tracked in my branches.
>>>>>> 
>>>>>> Drill: https://github.com/KulykRoman/drill/commits/
>>> CalciteForkRebase_rc1
>>>>>> Calcite:
>>>>>> https://github.com/KulykRoman/incubator-calcite/commits/
>>>>>> DrillCalcite1.13.0_rc1
>>>>>> 
>>>>>> Currently, I face ~160 errors in java-exec module unit tests. I have
>>>>>> divided them into seven groups:
>>>>>> 1) SYSTEM ERROR: ClassCastException
>>>>>> 2) Unexpected column errors
>>>>>> 3) Runtime Exceptions
>>>>>> 4) validation errors
>>>>>> 5) RpcExceptions (generalized group: some of them should be fixed
by
>>> the
>>>>>> previous groups)
>>>>>> 6) IllegalState errors (generalized group: some of them should be
>> fixed
>>>>> by
>>>>>> the previous group)
>>>>>> 7) Other errors
>>>>>> 
>>>>>> Best regards,
>>>>>> Roman Kulyk
>>>>>> 
>>>>> 
>>> 
>>> 
>> 


Mime
View raw message