drill-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Timothy Farkas <tfar...@mapr.com>
Subject Re: [DISCUSS] 1.14.0 release
Date Sat, 14 Jul 2018 01:36:19 GMT
Hi Boaz,

I looked at DRILL-6606 and have updated the ticket, I should have a fix
monday. It looks like a minor logical error.

I'm not clear on why you suspect DRILL-6453 is cause by batch sniffing,
perhaps we can discuss offline.

Thanks,
Tim

On Fri, Jul 13, 2018 at 2:01 PM, Arina Yelchiyeva <
arina.yelchiyeva@gmail.com> wrote:

> Two more regressions:
> https://urldefense.proofpoint.com/v2/url?u=https-3A__issues.
> apache.org_jira_browse_DRILL-2D6603&d=DwIBaQ&c=cskdkSMqhcnjZxdQVpwTXg&r=
> 4eQVr8zB8ZBff-yxTimdOQ&m=F7Ih5Ah_SfOS5fZXFApt88iMe3Vd-
> Jq1XDvxPN6b3y4&s=FqDR26vK2kVG-P69NcqoqNxRrdHKZvBCWamYRftPPYg&e=
> https://urldefense.proofpoint.com/v2/url?u=https-3A__issues.
> apache.org_jira_browse_DRILL-2D6605&d=DwIBaQ&c=cskdkSMqhcnjZxdQVpwTXg&r=
> 4eQVr8zB8ZBff-yxTimdOQ&m=F7Ih5Ah_SfOS5fZXFApt88iMe3Vd-Jq1XDvxPN6b3y4&s=
> KzhJcDP4F8FIA7h4GiQE8wg_CSae0CAI0yEMslff52o&e=
>
> Kind regards,
> Arina
>
> On Fri, Jul 13, 2018 at 11:25 PM Sorabh Hamirwasia <shamirwasia@mapr.com>
> wrote:
>
> > Hi Boaz,
> > Couple of updates.
> >
> > *Merged In:*
> > DRILL-6542: (May be Ready2Commit soon) IndexOutOfBounds exception for
> > multilevel lateral ((Sorabh / Parth))
> >
> > *In Review:*
> >
> >
> > *DRILL-6475: Query with UNNEST causes a Null Pointer .  (( Hanumath ))*
> > Thanks,
> > Sorabh
> >
> > On Fri, Jul 13, 2018 at 1:17 PM, Parth Chandra <parthc@apache.org>
> wrote:
> >
> > > Our (unwritten) rule has been that a commit cannot even go in unless
> unit
> > > _and_ regression tests pass.
> > > Releases are stricter, all tests, longevity tests, UI, are required to
> > > pass. In addition, any performance regression needs to be discussed.
> > >
> > > So far we have not made any exceptions, but that is not to say we
> cannot.
> > >
> > > On Fri, Jul 13, 2018 at 1:03 PM, Vlad Rozov <vrozov@apache.org> wrote:
> > >
> > > > My 2 cents:
> > > >
> > > > From Apache point of view it is OK to do a release even if unit tests
> > do
> > > > not pass at all or there is a large number of regression introduced.
> > > Apache
> > > > release is a source release and as long as it compiles and does not
> > have
> > > > license issues, it is up to community (PMC) to decide on any other
> > > criteria
> > > > for a release.
> > > >
> > > > The issue in DRILL-6453 is not limited to a large number of hash
> joins.
> > > It
> > > > should be possible to reproduce it even with a single hash join as
> long
> > > as
> > > > left and right sides are getting batches from one(many) to many
> > exchanges
> > > > (broadcast or hash partitioner senders).
> > > >
> > > > Thank you,
> > > >
> > > > Vlad
> > > >
> > > >
> > > > On 7/13/18 08:41, Aman Sinha wrote:
> > > >
> > > >> I would say we have to take a measured approach to this and decide
> on
> > a
> > > >> case-by-case which issue is a show stopper.
> > > >> While of course we have to make every effort to avoid regression,
we
> > > >> cannot
> > > >> claim that a particular release will not cause any regression.
> > > >> I believe there are 10000+ passing tests,  so that should provide
a
> > > level
> > > >> of confidence.   The TPC-DS 72 is a 10 table join which in the
> hadoop
> > > >> world
> > > >> of
> > > >> denormalized schemas is not relatively common.  The main question
is
> > > does
> > > >> the issue reproduce with fewer joins having the same type of
> > > distribution
> > > >> plan ?
> > > >>
> > > >>
> > > >> Aman
> > > >>
> > > >> On Fri, Jul 13, 2018 at 7:36 AM Arina Yelchiyeva <
> > > >> arina.yelchiyeva@gmail.com>
> > > >> wrote:
> > > >>
> > > >> We cannot release with existing regressions, especially taking into
> > > >>> account
> > > >>> the there are not minor issues.
> > > >>> As far as I understand reverting is not an option since hash join
> > spill
> > > >>> feature are extended into several commits + subsequent fixes.
> > > >>> I guess we need to consider postponing the release until issues
are
> > > >>> resolved.
> > > >>>
> > > >>> Kind regards,
> > > >>> Arina
> > > >>>
> > > >>> On Fri, Jul 13, 2018 at 5:14 PM Boaz Ben-Zvi <boaz@apache.org>
> > wrote:
> > > >>>
> > > >>> (Guessing ...) It is possible that the root cause for DRILL-6606
is
> > > >>>> similar to that in  DRILL-6453 -- that is the new "early sniffing"
> > in
> > > >>>> the
> > > >>>> Hash-Join, which repeatedly invokes next() on the two "children"
> of
> > > the
> > > >>>> join *during schema discovery* until non-empty data is returned
> (or
> > > >>>> NONE,
> > > >>>> STOP, etc).  Last night Salim, Vlad and I briefly discussed
> > > >>>> alternatives,
> > > >>>> like postponing the "sniffing" to a later time (beginning
of the
> > build
> > > >>>>
> > > >>> for
> > > >>>
> > > >>>> the right child, and beginning of the probe for the left child).
> > > >>>>
> > > >>>> However this would require some work time. So what should
we do
> > about
> > > >>>>
> > > >>> 1.14
> > > >>>
> > > >>>> ?
> > > >>>>
> > > >>>>    Thanks,
> > > >>>>
> > > >>>>            Boaz
> > > >>>>
> > > >>>> On Fri, Jul 13, 2018 at 3:46 AM, Arina Yelchiyeva <
> > > >>>> arina.yelchiyeva@gmail.com> wrote:
> > > >>>>
> > > >>>> During implementing late limit 0 optimization, Bohdan has
found
> one
> > > more
> > > >>>>> regression after Hash Join spill to disk.
> > > >>>>> https://urldefense.proofpoint.com/v2/url?u=https-3A__issues.
> > > apache.org_jira_browse_DRILL-2D6606&d=DwIBaQ&c=
> cskdkSMqhcnjZxdQVpwTXg&r=
> > > gRpEl0WzXE3EMrwj0KFbZXGXRyadOthF2jlYxvhTlQg&m=TGqnVoxNweQMiHTgP4J-
> > > rOnguFThVKShnQqHE_CmySI&s=aHUGrClgE_9UsRpRlNM95TbW91ivkqGdF1hV0EDc
> 3xU&e=
> > > >>>>> <
> > > >>>>>
> > > >>>> https://urldefense.proofpoint.com/v2/url?u=https-3A__issues.
> > > >>> apache.org_jira_browse_DRILL-2D6606&d=DwMFaQ&c=cskdkSMqhcnjZ
> > > >>> xdQVpwTXg&r=7lXQnf0aC8VQ0iMXwVgNHw&m=OHnyHeZpNk3hcwkG-JoQG6E
> > > >>> 90tKdoS47J1rv5x-hJzw&s=wm5zpJf9K2zYzrqRB1LqLpKcvmBK5y
> 6XC0ZUqVmSjko&e=
> > > >>>
> > > >>>> Boaz please take a look.
> > > >>>>>
> > > >>>>> Kind regards,
> > > >>>>> Arina
> > > >>>>>
> > > >>>>>
> > > >>>>
> > > >
> > >
> >
>

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message