drill-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Michael Mior <mm...@uwaterloo.ca>
Subject Re: Working on upgrade Drill Calcite version
Date Fri, 19 Jan 2018 14:46:30 GMT
Fantastic news! Thanks to all who made this happen.

--
Michael Mior
mmior@apache.org

2018-01-19 5:43 GMT-05:00 Arina Ielchiieva <arina@apache.org>:

> I am happy to announce that Apache Drill has rebased on Calcite 1.15. All
> information can be found in DRILL-3993 and in Google Doc [1].
> Currently changes are in master and are intended to be part of Drill 1.13.0
> release unless something critical pops up. Feel free to test them and give
> your feedback.
>
> Thanks Roman Kulyk and Volodymyr Vysotskyi for making this happen.
> Aman Sinha and Chunhui Shi for the code review.
> And of course Julian Hyde for helping to address / review Calcite specific
> issues.
>
> [1]
> https://docs.google.com/document/d/1Lqk9NoKQviz0YimBmov4z1pu
> i7QjJGjDVwMa1p0emPk/edit#heading=h.i3rowg20vxv4
>
>
> Kind regards
> Arina
>
> On Fri, Sep 1, 2017 at 8:01 PM, Roman Kulyk <rom.kulyk@gmail.com> wrote:
>
> > Team, thanks for your answers!
> >
> > Julian, сontinuing the topic of Calcite tests I just want to note that I
> > have ignored 7 test in 5ba68240 commit in Calcite. Here is a list of
> > ignored tests:
> >
> >   TraitConversionTest.testTraitConversion
> >   JdbcAdapterTest.testCrossJoinWithJoinKeyPlan
> >   JdbcAdapterTest.testCrossJoinWithJoinKeyAndFilterPlan
> >   JdbcTest.testJoinManyWay (it was also ignored by 8a87ad1e5)
> >   LatticeTest.testTileAlgorithm (it was also ignored by 2e076e1a)
> >   LatticeTest.testTileAlgorithm2 (it was also ignored by 8a87ad1e5)
> >   LatticeTest.testTwoLattices
> >
> > You can see detailed information in my comments. Also, I want to notice
> > that there are 2 required commits in my Calcite branch: 5ba68240 and
> > 1f5145b1. Without them, we lose a big part of Drill functionality.
> >
> > Best regards,
> > Roman Kulyk
> >
> > 2017-08-31 18:43 GMT+00:00 Julian Hyde <jhyde@apache.org>:
> >
> > > +dev@calcite.
> > >
> > > Excellent work, Roman.
> > >
> > > Your changes to Calcite don’t seem to be very major, and I ran the test
> > > suite on your https://github.com/KulykRoman/incubator-calcite/commits/
> > > DrillCalcite1.13.0_rc1 branch and all the tests pass. So when you are
> > > ready, I think we can expedite getting these changes back into Calcite.
> > >
> > > Calcite will release 1.14 probably in the next two weeks. So, when
> you’re
> > > on to 1.13 let’s continue the momentum and get up to the latest
> release.
> > I
> > > would love to get to the situation that we can ask Drill to verify a
> > > Calcite release before we make it. There are possibly breaking changes
> in
> > > the area of schema and grouping-sets aggregate but it should be mostly
> > > straightforward.
> > >
> > > Julian
> > >
> > >
> > > > On Aug 31, 2017, at 8:12 AM, Jinfeng Ni <jni@apache.org> wrote:
> > > >
> > > > It's great to hear that someone is working on moving Drill on new
> > Calcite
> > > > release! As someone who did similar job in last two rebase work, I
> > could
> > > > see that there would be many regressions to fix. Please let me know
> if
> > I
> > > > can provide any help.
> > > >
> > > > I thought we have pushed almost Drill specific commits to Calcite.
> I'll
> > > > take a look at your Calcite branch. Ideally, we want to get rid of
> all
> > > > Drill specific commits, after the rebase work this time.
> > > >
> > > > Thanks,
> > > >
> > > > Jinfeng
> > > >
> > > >
> > > >
> > > >
> > > > On Thu, Aug 31, 2017 at 8:04 AM, Aman Sinha <amansinha@apache.org>
> > > wrote:
> > > >
> > > >> Yes, this is long overdue !  Thanks for working on it Roman.   If
> > > needed,
> > > >> we can do a separate hangout with a few selected folks who have
> worked
> > > on
> > > >> Drill+Calcite to provide feedback on how to resolve the test
> failures.
> > > >> Although, at this stage you probably know a lot more about the
> > > integration
> > > >> than many of us.
> > > >>
> > > >> -Aman
> > > >>
> > > >> On Thu, Aug 31, 2017 at 7:33 AM, Roman Kulyk <rom.kulyk@gmail.com>
> > > wrote:
> > > >>
> > > >>> Hi, team!
> > > >>>
> > > >>> I want to inform that I am working on updating Drill Calcite
> version
> > to
> > > >>> 1.13. I forked from Calcite "branch-1.13" and added some commits:
> > > >>> - DRILL-1455: Add return type-inference strategy for arithmetic
> > > operators
> > > >>> when one of the arguments is ANY type. (was not in Calcite)
> > > >>> - Add new method to ViewExpander interface to allow passing
> > SchemaRoot.
> > > >>> (was not in Calcite)
> > > >>> - Allow a MAP literal type. (was not in Calcite)
> > > >>> - DRILL-4047: Modify table functions to allow querying a table
with
> > > >> options
> > > >>> in Drill (was not in Calcite)
> > > >>> - Drill-specific change: Add back AbstractConverter in RelSet.java
> > ...
> > > >>> (small changes after a3bc0d8 commit)
> > > >>> - CALCITE-628 related but not fix the problem: Ensure target traits
> > are
> > > >>> simple when use Frameworks or RelOptRule.convert() method. (small
> > > changes
> > > >>> after a3bc0d8 commit)
> > > >>> -  Support select * from schema-less table in execution engine
like
> > > Drill
> > > >>> (small changes after 9bd7d75 commit)
> > > >>>
> > > >>> Only after adding these commits to Calcite, a part of  Drill
> > > >> functionality
> > > >>> started to work as before. Currently, I am at Drill integration
> > stage.
> > > I
> > > >> am
> > > >>> fixing Drill unit tests. Progress can be tracked in my branches.
> > > >>>
> > > >>> Drill: https://github.com/KulykRoman/drill/commits/
> > > CalciteForkRebase_rc1
> > > >>> Calcite:
> > > >>> https://github.com/KulykRoman/incubator-calcite/commits/
> > > >>> DrillCalcite1.13.0_rc1
> > > >>>
> > > >>> Currently, I face ~160 errors in java-exec module unit tests.
I
> have
> > > >>> divided them into seven groups:
> > > >>> 1) SYSTEM ERROR: ClassCastException
> > > >>> 2) Unexpected column errors
> > > >>> 3) Runtime Exceptions
> > > >>> 4) validation errors
> > > >>> 5) RpcExceptions (generalized group: some of them should be fixed
> by
> > > the
> > > >>> previous groups)
> > > >>> 6) IllegalState errors (generalized group: some of them should
be
> > fixed
> > > >> by
> > > >>> the previous group)
> > > >>> 7) Other errors
> > > >>>
> > > >>> Best regards,
> > > >>> Roman Kulyk
> > > >>>
> > > >>
> > >
> > >
> >
>

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message