trafodion-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Qifan Chen <qifan.c...@esgyn.com>
Subject Re: Proposal to add hive regression tests to check-PR tests
Date Wed, 27 Jul 2016 16:21:08 GMT
Sure, TEST030 can be included which can finish in 1 minute.

TEST009 tests external tables, and TEST030 has similar tests.  Maybe
exclude TEST009?

TEST001 should be included since it tests both hive/text and hive/ORC.

--Qifan



On Wed, Jul 27, 2016 at 11:09 AM, Sandhya Sundaresan <
sandhya.sundaresan@esgyn.com> wrote:

> Agree TEST018 is definitely a candidate. Even installer changes have an
> effect on that test.
> Will let other experts chime in for more suggestions .
> Sandhya
>
> -----Original Message-----
> From: Roberta Marton [mailto:roberta.marton@esgyn.com]
> Sent: Wednesday, July 27, 2016 8:52 AM
> To: dev@trafodion.incubator.apache.org
> Subject: RE: Proposal to add hive regression tests to check-PR tests
>
> Perhaps we should choose tests that fail most frequently instead? Tests
> 009,
> 018, and 030 comes to mind.
>
>       Roberta
>
> -----Original Message-----
> From: Qifan Chen [mailto:qifan.chen@esgyn.com]
> Sent: Wednesday, July 27, 2016 6:36 AM
> To: dev <dev@trafodion.incubator.apache.org>
> Subject: Re: Proposal to add hive regression tests to check-PR tests
>
> Here is a tally of run-time (in minutes) of one run of the hive regression.
>
> TEST1 1
> TEST2 1
> TEST3 3
> TEST5 7
> TEST6 1
> TEST9 4
> TEST15 5
> TEST17 5
> TEST18 6
> TEST21 1
> TEST30 1
> TEST31 1
> TEST33 4
> TEST34 2
> TEST35 3
> TEST36 1
>
> The total time used is about 46 minutes.
>
> My vote will be to include a subset of from the above list what are fast to
> run (say all 1 minute ones).  It probably will be a good idea to keep them
> still in hive tests.
>
> Thanks
>
> --Qifan
>
>
> On Tue, Jul 26, 2016 at 5:37 PM, Selva Govindarajan <
> selva.govindarajan@esgyn.com> wrote:
>
> > Thanks Steve for resurrecting this discussion.
> >
> > Hive tests have been stabilized to a greater extent that we shouldn't
> > have false failures now. Recently, there has been a quite amount of
> > contribution coming in the area related to hive in Trafodion.  Hence I
> > would vote +1 for adding hive tests to check PR.
> >
> > Selva
> >
> > -----Original Message-----
> > From: Steve Varnau [mailto:steve.varnau@esgyn.com]
> > Sent: Tuesday, July 26, 2016 1:12 PM
> > To: dev@trafodion.incubator.apache.org
> > Subject: RE: Proposal to add hive regression tests to check-PR tests
> >
> > Hi,
> >
> > I wanted to revisit this discussion to come to resolution.  There was
> > a digression into the idea of dynamically choosing tests, but I'd like
> > to come back to original proposal of adding an extra suite to the
> > check tests.
> >
> > As I read the thread, there were several responses in support of the
> > proposal, and a couple of reservations. The reservations include
> > increasing the chance for false failures, which already can be a
> > headache. Also the concern of adding long running tests that are
> > included in hive versus maybe adding a few more small tests to core.
> > Or perhaps using "extra tests" as needed, which is available on request.
> >
> > I'm willing to add another test job if that is what the community
> > wants, but might it make more sense to more small tests to core or
> > move some from hive to core?
> >
> > --Steve
> >
> > > -----Original Message-----
> > > From: Selva Govindarajan [mailto:selva.govindarajan@esgyn.com]
> > > Sent: Monday, July 18, 2016 10:28 PM
> > > To: dev@trafodion.incubator.apache.org
> > > Subject: RE: Proposal to add hive regression tests to check-PR tests
> > >
> > > I totally agree with Steve to use a simple and predictable mechanism
> > > to do check PR tests, If my memory serves me right, prior to
> > > Trafodion becoming an Apache incubating project, hive tests were
> > > part of check-PR. Because of unpredictable state of hive regressions
> > > then it was decided to suspend
> > > running hive regressions as part of check-in.   Based on the current
> > state
> > > of Trafodion, and the fact that the hive regressions have been
> > > stabilized to a greater extent, it is important that this stability
> > > is maintained by the future contributions. Recently many
> > > contributions have come in hive-related area of the code.
> > >
> > > Adding hive regressions as part of check-PR should not increase the
> > > overall time to complete the check-PR, but it would require
> > > additional resources,.
> > > Hence, Trafodion Jenkins infrastructure would incur additional cost.
> > >
> > > I am expecting the Trafodion Release Manager of R2.1 will help us to
> > > determine with the community input what is the best option to go with.
> > >
> > > Selva
> > > -----Original Message-----
> > > From: Steve Varnau [mailto:steve.varnau@esgyn.com]
> > > Sent: Monday, July 18, 2016 12:27 PM
> > > To: dev@trafodion.incubator.apache.org
> > > Subject: RE: Proposal to add hive regression tests to check-PR tests
> > >
> > > The current test process looks at which files have been modified and
> > > puts it into a bucket, which is used to determine what tests to run.
> > > However, the only buckets that now exist are DOC and NONDOC.
> > >
> > > So if the change consists only of things in the docs/ tree, then it
> > > only does static check and a docs build.  If there are non-docs
> > > changes, it assumes it needs to run all the build and tests.
> > >
> > > It is pretty conservative, but the more heuristics we put in to
> > > customize the tests, the more chance that it will miss something.  I
> > > can imagine a connectivity only change not running the jobs that
> > > don't exercise connectivity. But figuring out what things might
> > > affect hive tests seems much harder.
> > >
> > > There are many things (installer, executor,...) that might affect
> > > any of our tests.  Seems safer to keep the test heuristics very
> > > simple and predictable, and change the content of the test suites to
> > > what ought to be in check versus nightly.
> > >
> > > --Steve
> > >
> > > > -----Original Message-----
> > > > From: Qifan Chen [mailto:qifan.chen@esgyn.com]
> > > > Sent: Monday, July 18, 2016 9:43 AM
> > > > To: dev <dev@trafodion.incubator.apache.org>
> > > > Subject: Re: Proposal to add hive regression tests to check-PR
> > > > tests
> > > >
> > > > The author just honestly describes the changes, and the tool picks
> > > > the right tests.  Thanks --Qifan
> > > >
> > > > On Mon, Jul 18, 2016 at 11:29 AM, Sean Broeder
> > > > <sean.broeder@esgyn.com>
> > > > wrote:
> > > >
> > > > > I'd prefer not to leave it up to authors to select which tests
> > > > > are appropriate.  Sometimes we get it right and others we are
> > > > > horribly wrong.
> > > > >
> > > > > Thanks,
> > > > > Sean
> > > > >
> > > > > -----Original Message-----
> > > > > From: Qifan Chen [mailto:qifan.chen@esgyn.com]
> > > > > Sent: Monday, July 18, 2016 9:20 AM
> > > > > To: dev <dev@trafodion.incubator.apache.org>
> > > > > Subject: Re: Proposal to add hive regression tests to check-PR
> > > > > tests
> > > > >
> > > > > I agree with Sandhya and wonder if we can enhance check-PR tests
> > > > > (hive
> > > > for
> > > > > example, in question) with the following twist.
> > > > >
> > > > >    1. Randomly select several (say 3) tests from regression/hive.
> > > > > The
> > > > >    rational is that we only need to  sanity check the changes
> > > > > and a full daily
> > > > >    build with test will follow the merge.
> > > > >    2. Before the check-in, we always run the full regression
> > > > > test, and I do
> > > > >    not see the value to run full Hive again in check-PR.
> > > > >    3. In the future, we could find the most appreciate tests for
> > > > > check-PR
> > > > >    (instead of randomly select, or select the full set).  The
> > > > > author can point
> > > > >    out the nature of the change and the check-in tool does the
> > > > > selection.
> > > > > For
> > > > >    example, a change in DoP for Hbase tables will select some
> > > > > tests from
> > > > >    regress/seabase, but not from regress/hive.
> > > > >
> > > > > Thanks
> > > > >
> > > > > --Qifan
> > > > >
> > > > > On Mon, Jul 18, 2016 at 10:46 AM, Sandhya Sundaresan <
> > > > > sandhya.sundaresan@esgyn.com> wrote:
> > > > >
> > > > > > +0 for me.
> > > > > >  I am not sure of the need to add the whole test suite  to
> > > > > > check tests.
> > > > > > The hive regressions do run nightly anyway so failures should
> > > > > > be clear on each nightly run on a daily basis.
> > > > > > My concern is that long running tests like hive/TEST018   are
> more
> > > > > > to
> > > > > test
> > > > > > features like bulkload/unload and since we already have the
> > > > > > option to run "extra tests" in Jenkins, I'm not sure bringing
> > > > > > in entire test suites  into check tests is the right approach
> > > > > > or trend going forward and  adding time and resources to what
> > > > > > is supposed to be a sanity test
> > > > > for
> > > > > > every single  PR.
> > > > > >
> > > > > > Sandhya
> > > > > >
> > > > > > -----Original Message-----
> > > > > > From: Selva Govindarajan [mailto:selva.govindarajan@esgyn.com]
> > > > > > Sent: Monday, July 18, 2016 7:22 AM
> > > > > > To: dev@trafodion.incubator.apache.org
> > > > > > Subject: RE: Proposal to add hive regression tests to check-PR
> > > > > > tests
> > > > > >
> > > > > > Hive regressions takes little less than an hour. As I said
> > > > > > before, the time is not a factor because the regressions are
> > > > > > run in parallel in different VMs. Seabase regressions which
is
> > > > > > run as part of check-PR takes around 1 hour and 40 mins. Hence
> > > > > > hive regressions shouldn't add more time for check-PR to
> > > > > > complete, but of course it would need another VM.
> > > > > >
> > > > > > Selva
> > > > > >
> > > > > > -----Original Message-----
> > > > > > From: Jin, Jian (Seth) [mailto:Jian.jin@esgyn.cn]
> > > > > > Sent: Sunday, July 17, 2016 7:31 PM
> > > > > > To: dev@trafodion.incubator.apache.org
> > > > > > Subject: RE: Proposal to add hive regression tests to check-PR
> > > > > > tests
> > > > > >
> > > > > > How long will it take for Hive regression?
> > > > > >
> > > > > > Br,
> > > > > >
> > > > > > Seth
> > > > > >
> > > > > > -----Original Message-----
> > > > > > From: Liu, Ming (Ming) [mailto:ming.liu@esgyn.cn]
> > > > > > Sent: 2016年7月16日 9:16
> > > > > > To: dev@trafodion.incubator.apache.org
> > > > > > Subject: RE: Proposal to add hive regression tests to check-PR
> > > > > > tests
> > > > > >
> > > > > > +1 to this
> > > > > >
> > > > > > -----Original Message-----
> > > > > > From: Selva Govindarajan [mailto:selva.govindarajan@esgyn.com]
> > > > > > Sent: Saturday, July 16, 2016 9:08 AM
> > > > > > To: dev@trafodion.incubator.apache.org
> > > > > > Subject: Proposal to add hive regression tests to check-PR
> > > > > > tests
> > > > > >
> > > > > > If you have subscribed to Trafodion Daily Build, you would
> > > > > > have noticed that the daily build has been failing for some
days.
> > > > > > Most often, it is due the failure in hive regression tests run
> > > > > > as part of the daily build.
> > > > > > Lately, there has been some conscious effort made successfully
> > > > > > to ensure that the hive regression tests can be run reliably.
> > > > > > To maintain the Trafodion daily build in that state, I am
> > > > > > proposing to include hive regressions to check-PR tests.  It
> > > > > > shouldn’t add the overall time taken to regressions tests
> > > > > > because tests are run in parallel on different VMs, though it
> > > > > > would consume more resources.
> > > > > >
> > > > > >
> > > > > >
> > > > > > -          Selva
> > > > > >
> > > > >
> > > > >
> > > > >
> > > > > --
> > > > > Regards, --Qifan
> > > > >
> > > >
> > > >
> > > >
> > > > --
> > > > Regards, --Qifan
> >
>
>
>
> --
> Regards, --Qifan
>



-- 
Regards, --Qifan

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message