drill-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Arina Ielchiieva <ar...@apache.org>
Subject Re: Working on upgrade Drill Calcite version
Date Fri, 19 Jan 2018 10:43:15 GMT
I am happy to announce that Apache Drill has rebased on Calcite 1.15. All
information can be found in DRILL-3993 and in Google Doc [1].
Currently changes are in master and are intended to be part of Drill 1.13.0
release unless something critical pops up. Feel free to test them and give
your feedback.

Thanks Roman Kulyk and Volodymyr Vysotskyi for making this happen.
Aman Sinha and Chunhui Shi for the code review.
And of course Julian Hyde for helping to address / review Calcite specific
issues.

[1]
https://docs.google.com/document/d/1Lqk9NoKQviz0YimBmov4z1pui7QjJGjDVwMa1p0emPk/edit#heading=h.i3rowg20vxv4


Kind regards
Arina

On Fri, Sep 1, 2017 at 8:01 PM, Roman Kulyk <rom.kulyk@gmail.com> wrote:

> Team, thanks for your answers!
>
> Julian, сontinuing the topic of Calcite tests I just want to note that I
> have ignored 7 test in 5ba68240 commit in Calcite. Here is a list of
> ignored tests:
>
>   TraitConversionTest.testTraitConversion
>   JdbcAdapterTest.testCrossJoinWithJoinKeyPlan
>   JdbcAdapterTest.testCrossJoinWithJoinKeyAndFilterPlan
>   JdbcTest.testJoinManyWay (it was also ignored by 8a87ad1e5)
>   LatticeTest.testTileAlgorithm (it was also ignored by 2e076e1a)
>   LatticeTest.testTileAlgorithm2 (it was also ignored by 8a87ad1e5)
>   LatticeTest.testTwoLattices
>
> You can see detailed information in my comments. Also, I want to notice
> that there are 2 required commits in my Calcite branch: 5ba68240 and
> 1f5145b1. Without them, we lose a big part of Drill functionality.
>
> Best regards,
> Roman Kulyk
>
> 2017-08-31 18:43 GMT+00:00 Julian Hyde <jhyde@apache.org>:
>
> > +dev@calcite.
> >
> > Excellent work, Roman.
> >
> > Your changes to Calcite don’t seem to be very major, and I ran the test
> > suite on your https://github.com/KulykRoman/incubator-calcite/commits/
> > DrillCalcite1.13.0_rc1 branch and all the tests pass. So when you are
> > ready, I think we can expedite getting these changes back into Calcite.
> >
> > Calcite will release 1.14 probably in the next two weeks. So, when you’re
> > on to 1.13 let’s continue the momentum and get up to the latest release.
> I
> > would love to get to the situation that we can ask Drill to verify a
> > Calcite release before we make it. There are possibly breaking changes in
> > the area of schema and grouping-sets aggregate but it should be mostly
> > straightforward.
> >
> > Julian
> >
> >
> > > On Aug 31, 2017, at 8:12 AM, Jinfeng Ni <jni@apache.org> wrote:
> > >
> > > It's great to hear that someone is working on moving Drill on new
> Calcite
> > > release! As someone who did similar job in last two rebase work, I
> could
> > > see that there would be many regressions to fix. Please let me know if
> I
> > > can provide any help.
> > >
> > > I thought we have pushed almost Drill specific commits to Calcite. I'll
> > > take a look at your Calcite branch. Ideally, we want to get rid of all
> > > Drill specific commits, after the rebase work this time.
> > >
> > > Thanks,
> > >
> > > Jinfeng
> > >
> > >
> > >
> > >
> > > On Thu, Aug 31, 2017 at 8:04 AM, Aman Sinha <amansinha@apache.org>
> > wrote:
> > >
> > >> Yes, this is long overdue !  Thanks for working on it Roman.   If
> > needed,
> > >> we can do a separate hangout with a few selected folks who have worked
> > on
> > >> Drill+Calcite to provide feedback on how to resolve the test failures.
> > >> Although, at this stage you probably know a lot more about the
> > integration
> > >> than many of us.
> > >>
> > >> -Aman
> > >>
> > >> On Thu, Aug 31, 2017 at 7:33 AM, Roman Kulyk <rom.kulyk@gmail.com>
> > wrote:
> > >>
> > >>> Hi, team!
> > >>>
> > >>> I want to inform that I am working on updating Drill Calcite version
> to
> > >>> 1.13. I forked from Calcite "branch-1.13" and added some commits:
> > >>> - DRILL-1455: Add return type-inference strategy for arithmetic
> > operators
> > >>> when one of the arguments is ANY type. (was not in Calcite)
> > >>> - Add new method to ViewExpander interface to allow passing
> SchemaRoot.
> > >>> (was not in Calcite)
> > >>> - Allow a MAP literal type. (was not in Calcite)
> > >>> - DRILL-4047: Modify table functions to allow querying a table with
> > >> options
> > >>> in Drill (was not in Calcite)
> > >>> - Drill-specific change: Add back AbstractConverter in RelSet.java
> ...
> > >>> (small changes after a3bc0d8 commit)
> > >>> - CALCITE-628 related but not fix the problem: Ensure target traits
> are
> > >>> simple when use Frameworks or RelOptRule.convert() method. (small
> > changes
> > >>> after a3bc0d8 commit)
> > >>> -  Support select * from schema-less table in execution engine like
> > Drill
> > >>> (small changes after 9bd7d75 commit)
> > >>>
> > >>> Only after adding these commits to Calcite, a part of  Drill
> > >> functionality
> > >>> started to work as before. Currently, I am at Drill integration
> stage.
> > I
> > >> am
> > >>> fixing Drill unit tests. Progress can be tracked in my branches.
> > >>>
> > >>> Drill: https://github.com/KulykRoman/drill/commits/
> > CalciteForkRebase_rc1
> > >>> Calcite:
> > >>> https://github.com/KulykRoman/incubator-calcite/commits/
> > >>> DrillCalcite1.13.0_rc1
> > >>>
> > >>> Currently, I face ~160 errors in java-exec module unit tests. I have
> > >>> divided them into seven groups:
> > >>> 1) SYSTEM ERROR: ClassCastException
> > >>> 2) Unexpected column errors
> > >>> 3) Runtime Exceptions
> > >>> 4) validation errors
> > >>> 5) RpcExceptions (generalized group: some of them should be fixed by
> > the
> > >>> previous groups)
> > >>> 6) IllegalState errors (generalized group: some of them should be
> fixed
> > >> by
> > >>> the previous group)
> > >>> 7) Other errors
> > >>>
> > >>> Best regards,
> > >>> Roman Kulyk
> > >>>
> > >>
> >
> >
>

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message