drill-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Arina Yelchiyeva <arina.yelchiy...@gmail.com>
Subject Re: [DISCUSS] 1.14.0 release
Date Fri, 13 Jul 2018 21:01:05 GMT
Two more regressions:
https://issues.apache.org/jira/browse/DRILL-6603
https://issues.apache.org/jira/browse/DRILL-6605

Kind regards,
Arina

On Fri, Jul 13, 2018 at 11:25 PM Sorabh Hamirwasia <shamirwasia@mapr.com>
wrote:

> Hi Boaz,
> Couple of updates.
>
> *Merged In:*
> DRILL-6542: (May be Ready2Commit soon) IndexOutOfBounds exception for
> multilevel lateral ((Sorabh / Parth))
>
> *In Review:*
>
>
> *DRILL-6475: Query with UNNEST causes a Null Pointer .  (( Hanumath ))*
> Thanks,
> Sorabh
>
> On Fri, Jul 13, 2018 at 1:17 PM, Parth Chandra <parthc@apache.org> wrote:
>
> > Our (unwritten) rule has been that a commit cannot even go in unless unit
> > _and_ regression tests pass.
> > Releases are stricter, all tests, longevity tests, UI, are required to
> > pass. In addition, any performance regression needs to be discussed.
> >
> > So far we have not made any exceptions, but that is not to say we cannot.
> >
> > On Fri, Jul 13, 2018 at 1:03 PM, Vlad Rozov <vrozov@apache.org> wrote:
> >
> > > My 2 cents:
> > >
> > > From Apache point of view it is OK to do a release even if unit tests
> do
> > > not pass at all or there is a large number of regression introduced.
> > Apache
> > > release is a source release and as long as it compiles and does not
> have
> > > license issues, it is up to community (PMC) to decide on any other
> > criteria
> > > for a release.
> > >
> > > The issue in DRILL-6453 is not limited to a large number of hash joins.
> > It
> > > should be possible to reproduce it even with a single hash join as long
> > as
> > > left and right sides are getting batches from one(many) to many
> exchanges
> > > (broadcast or hash partitioner senders).
> > >
> > > Thank you,
> > >
> > > Vlad
> > >
> > >
> > > On 7/13/18 08:41, Aman Sinha wrote:
> > >
> > >> I would say we have to take a measured approach to this and decide on
> a
> > >> case-by-case which issue is a show stopper.
> > >> While of course we have to make every effort to avoid regression, we
> > >> cannot
> > >> claim that a particular release will not cause any regression.
> > >> I believe there are 10000+ passing tests,  so that should provide a
> > level
> > >> of confidence.   The TPC-DS 72 is a 10 table join which in the hadoop
> > >> world
> > >> of
> > >> denormalized schemas is not relatively common.  The main question is
> > does
> > >> the issue reproduce with fewer joins having the same type of
> > distribution
> > >> plan ?
> > >>
> > >>
> > >> Aman
> > >>
> > >> On Fri, Jul 13, 2018 at 7:36 AM Arina Yelchiyeva <
> > >> arina.yelchiyeva@gmail.com>
> > >> wrote:
> > >>
> > >> We cannot release with existing regressions, especially taking into
> > >>> account
> > >>> the there are not minor issues.
> > >>> As far as I understand reverting is not an option since hash join
> spill
> > >>> feature are extended into several commits + subsequent fixes.
> > >>> I guess we need to consider postponing the release until issues are
> > >>> resolved.
> > >>>
> > >>> Kind regards,
> > >>> Arina
> > >>>
> > >>> On Fri, Jul 13, 2018 at 5:14 PM Boaz Ben-Zvi <boaz@apache.org>
> wrote:
> > >>>
> > >>> (Guessing ...) It is possible that the root cause for DRILL-6606 is
> > >>>> similar to that in  DRILL-6453 -- that is the new "early sniffing"
> in
> > >>>> the
> > >>>> Hash-Join, which repeatedly invokes next() on the two "children"
of
> > the
> > >>>> join *during schema discovery* until non-empty data is returned
(or
> > >>>> NONE,
> > >>>> STOP, etc).  Last night Salim, Vlad and I briefly discussed
> > >>>> alternatives,
> > >>>> like postponing the "sniffing" to a later time (beginning of the
> build
> > >>>>
> > >>> for
> > >>>
> > >>>> the right child, and beginning of the probe for the left child).
> > >>>>
> > >>>> However this would require some work time. So what should we do
> about
> > >>>>
> > >>> 1.14
> > >>>
> > >>>> ?
> > >>>>
> > >>>>    Thanks,
> > >>>>
> > >>>>            Boaz
> > >>>>
> > >>>> On Fri, Jul 13, 2018 at 3:46 AM, Arina Yelchiyeva <
> > >>>> arina.yelchiyeva@gmail.com> wrote:
> > >>>>
> > >>>> During implementing late limit 0 optimization, Bohdan has found
one
> > more
> > >>>>> regression after Hash Join spill to disk.
> > >>>>> https://urldefense.proofpoint.com/v2/url?u=https-3A__issues.
> > apache.org_jira_browse_DRILL-2D6606&d=DwIBaQ&c=cskdkSMqhcnjZxdQVpwTXg&r=
> > gRpEl0WzXE3EMrwj0KFbZXGXRyadOthF2jlYxvhTlQg&m=TGqnVoxNweQMiHTgP4J-
> > rOnguFThVKShnQqHE_CmySI&s=aHUGrClgE_9UsRpRlNM95TbW91ivkqGdF1hV0EDc3xU&e=
> > >>>>> <
> > >>>>>
> > >>>> https://urldefense.proofpoint.com/v2/url?u=https-3A__issues.
> > >>> apache.org_jira_browse_DRILL-2D6606&d=DwMFaQ&c=cskdkSMqhcnjZ
> > >>> xdQVpwTXg&r=7lXQnf0aC8VQ0iMXwVgNHw&m=OHnyHeZpNk3hcwkG-JoQG6E
> > >>> 90tKdoS47J1rv5x-hJzw&s=wm5zpJf9K2zYzrqRB1LqLpKcvmBK5y6XC0ZUqVmSjko&e=
> > >>>
> > >>>> Boaz please take a look.
> > >>>>>
> > >>>>> Kind regards,
> > >>>>> Arina
> > >>>>>
> > >>>>>
> > >>>>
> > >
> >
>

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message