trafodion-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Eric Owhadi <eric.owh...@esgyn.com>
Subject Re: Where are the logs for the first set of regression tests?
Date Tue, 04 Aug 2015 22:36:47 GMT
OK clear,
Thanks again
Eric


On Tue, Aug 4, 2015 at 5:34 PM, Suresh Subbiah <suresh.subbiah60@gmail.com>
wrote:

> Hi Eric,
>
> Standard procedure is to modify the EXPECTEDaaa file. If you are satisfied
> with your new results you could simply copy the LOGaaa file as the new
> EXPECTEDaaa file. There are filters in the test suite that will hide things
> like timestamp from the final comparison.
> DIFFaaa.KNOWN files are meant to be temporary. To allow a test to pass
> while a fix is being worked on for a particular failure.
>
> Thanks
> Suresh
>
> On Tue, Aug 4, 2015 at 4:52 PM, Eric Owhadi <eric.owhadi@esgyn.com> wrote:
>
> > Thanks Sandhya, very useful!
> > BTW, if I want to change a test to take into account valid changes that
> > translate into false failure, should I modify the comparison file, or the
> > file that logs known legitimate differences?
> > Thanks again for the help :-),
> > Eric
> >
> > -----Original Message-----
> > From: Sandhya Sundaresan [mailto:sandhya.sundaresan@esgyn.com]
> > Sent: Tuesday, August 4, 2015 1:19 PM
> > To: dev@trafodion.incubator.apache.org
> > Subject: RE: Where are the logs for the first set of regression tests?
> >
> >  RE: Where are the logs for the first set of regression tests?
> >
> > There are logs and diffs. I notice I added the instructions on the wiki
> > only
> > for the individual "pull request" automated jobs. I will add them for the
> > official test results as well.
> >
> > Here they are :
> >
> > Click on the link in the daily build email :
> >
> > *https://jenkins02.trafodion.org/job/Check-Daily/48/*
> > <https://jenkins02.trafodion.org/job/Check-Daily/48/>
> >
> > ==> Click on "Console Output" for any test suite you want to check
> >
> > ==> Around the 5th or 6th line you will see a link to detailed logs .
> Click
> > that.
> >
> >
> >
> >    2015-08-04 08:30:13 Started by upstream project "Check-Daily" build
> >       number 48
> >
> >       2015-08-04 08:30:13 originally caused by:
> >
> >       2015-08-04 08:30:13  Started by timer
> >
> >       2015-08-04 08:30:13 [EnvInject] - Loading node environment
> variables.
> >
> >       2015-08-04 08:30:13 Building remotely on ahw22-172.16.0.75
> >       (redhat_ahw2.2) in workspace
> >       /home/jenkins/workspace/core-regress-fullstack2-ahw2.2
> >
> >       2015-08-04 08:30:14 [core-regress-fullstack2-ahw2.2] $ /bin/sh
> >       /tmp/hudson2056470402644332566.sh
> >
> >       2015-08-04 08:30:14 Detailed logs:
> >       *
> >
> http://logs.trafodion.org/Daily/2015-08-04_08-30-00/regress-fullstack2-ahw2.2/*
> >       <
> >
> http://logs.trafodion.org/Daily/2015-08-04_08-30-00/regress-fullstack2-ahw2.2/
> > >
> >
> >         == > Navigate to sql-regress-log
> >
> >                 You will see a directory for the component  test suite –
> eg
> > “executor” and a full  log – “Regress.log”.
> >
> >                 If you navigate to the component directory eg executor,
> you
> > will see all the DIFFs and LOG files .
> >
> > Sandhya
> >
> > -----Original Message-----
> > From: Eric Owhadi [mailto:eric.owhadi@esgyn.com <eric.owhadi@esgyn.com>]
> > Sent: Tuesday, August 4, 2015 11:06 AM
> > To: dev@trafodion.incubator.apache.org
> > Subject: RE: Where are the logs for the first set of regression tests?
> >
> > Hi Sandhya,
> >
> > Thanks for the offer, I'll first do a re-run with the feature off and get
> > familiar with the official results, that'll give me some learning
> > opportunity :-).
> >
> > There is no "diff" file on the official results automatic daily run? So
> > that
> > it is easy to compare with the failure one get running the test manually?
> >
> > Eric
> >
> > -----Original Message-----
> >
> > From: Sandhya Sundaresan [mailto:sandhya.sundaresan@esgyn.com
> > <sandhya.sundaresan@esgyn.com>]
> >
> > Sent: Tuesday, August 4, 2015 12:50 PM
> >
> > To: dev@trafodion.incubator.apache.org
> >
> > Subject: RE: Where are the logs for the first set of regression tests?
> >
> > Hi Eric,
> >
> >  They are run on a nightly basis. Attached are the results form last
> night
> > and previous night. As you can see 2 failures last night and both are
> being
> > investigated. Otherwise we have been having good clean runs. So any other
> > failures you see need to be investigated. If you can give me VNC access
> to
> > your machine, I can take a quick  look.
> >
> > I have documented how to read the official results on the wiki :
> >
> >
> >
> https://wiki.trafodion.org/wiki/index.php/Contributor_Workflow#Checking_Test_Results
> >
> > Sandhya
> >
> > -----Original Message-----
> >
> > From: Eric Owhadi [mailto:eric.owhadi@esgyn.com <eric.owhadi@esgyn.com>]
> >
> > Sent: Tuesday, August 4, 2015 10:40 AM
> >
> > To: dev@trafodion.incubator.apache.org
> >
> > Subject: RE: Where are the logs for the first set of regression tests?
> >
> > Out of curiosity, anyone have recently run the regression test all on
> > master
> > branch? I am getting some failure, and I wonder if they are linked with
> my
> > changes or if the current regression suit is having known temporary
> issues?
> >
> > Eric
> >
> > -----Original Message-----
> >
> > From: Dave Birdsall [mailto:dave.birdsall@esgyn.com
> > <dave.birdsall@esgyn.com>]
> >
> > Sent: Tuesday, August 4, 2015 11:26 AM
> >
> > To: dev@trafodion.incubator.apache.org
> >
> > Subject: RE: Where are the logs for the first set of regression tests?
> >
> > Right. Start with the simple stuff I think. Eric's e-mail trail is
> valuable
> > as it is fresh newbie experience; if we could answer Eric's questions on
> > the
> > wiki I think that would be a great start.
> >
> > -----Original Message-----
> >
> > From: Sandhya Sundaresan [mailto:sandhya.sundaresan@esgyn.com
> > <sandhya.sundaresan@esgyn.com>]
> >
> > Sent: Tuesday, August 4, 2015 9:24 AM
> >
> > To: dev@trafodion.incubator.apache.org
> >
> > Subject: RE: Where are the logs for the first set of regression tests?
> >
> > Hi Eric, Dave,
> >
> >   I can  add a page to the wiki for this.  We did have the steps
> documented
> > in our old internal wiki for those starting on Trafodion for the first
> > time.
> >
> > Also Roberta had a comprehensive  document describing the whole regress
> > environment back in MX days , I think. Not sure if anyone has that saved.
> >
> > Either way, for the wiki, perhaps a smaller page would be sufficient.
> >
> > Thanks
> >
> > Sandhya
> >
> > -----Original Message-----
> >
> > From: Eric Owhadi [mailto:eric.owhadi@esgyn.com <eric.owhadi@esgyn.com>]
> >
> > Sent: Tuesday, August 4, 2015 9:03 AM
> >
> > To: dev@trafodion.incubator.apache.org
> >
> > Subject: RE: Where are the logs for the first set of regression tests?
> >
> > I didn't want to ask, but indeed that would be fantastic and time saving.
> >
> > Also how to write and/or modify when tests are complaining, but we know
> > that
> > this is just because regression is based on delta, and cannot recognize
> > legitimate expected delta (due to know feature we just implemented), and
> > real regression.
> >
> > Eric
> >
> > -----Original Message-----
> >
> > From: Dave Birdsall [mailto:dave.birdsall@esgyn.com
> > <dave.birdsall@esgyn.com>]
> >
> > Sent: Tuesday, August 4, 2015 10:21 AM
> >
> > To: dev@trafodion.incubator.apache.org
> >
> > Subject: RE: Where are the logs for the first set of regression tests?
> >
> > Do we have a page on our Apache Trafodion wiki that describes how to run
> > the
> > regression tests and how to interpret and deal with the results? If
> someone
> > would like to volunteer to write such a page, it would be very useful to
> > newbies, I think.
> >
> > -----Original Message-----
> >
> > From: Eric Owhadi [mailto:eric.owhadi@esgyn.com <eric.owhadi@esgyn.com>]
> >
> > Sent: Tuesday, August 4, 2015 7:45 AM
> >
> > To: dev@trafodion.incubator.apache.org
> >
> > Subject: RE: Where are the logs for the first set of regression tests?
> >
> > Thanks Suresh, that's it indeed :-). Was driving me crazy.
> >
> > Eric
> >
> > -----Original Message-----
> >
> > From: Suresh Subbiah [mailto:suresh.subbiah60@gmail.com
> > <suresh.subbiah60@gmail.com>]
> >
> > Sent: Tuesday, August 4, 2015 9:41 AM
> >
> > To: dev@trafodion.incubator.apache.org
> >
> > Subject: Re: Where are the logs for the first set of regression tests?
> >
> > Hi Eric,
> >
> > I think the "first" set of tests is the same as the last set. This is an
> > artifact of how runallsb prints out results. At the end of every suite a
> > summary of that suite is printed out (with no header what suite it refers
> > to
> > since we have been running that suite for the last X minutes). Then at
> the
> > end of a run it concatenates the summary of every suite, this time with a
> > header for each suite. So if you kick off regressions now and come back
> > several hours later there will be a screen with a summary for all suites
> > and
> > then before the summary for core there will be list of results with no
> > suite
> > name. This is the same as the last suite in the summary (the hive test
> > suite).
> >
> > Thanks
> >
> > Suresh
> >
> > On Tue, Aug 4, 2015 at 9:20 AM, Qifan Chen <qifan.chen@esgyn.com> wrote:
> >
> > > Hi Eric,
> >
> > >
> >
> > > The results should be in one of the directories under $rundir.  For
> >
> > > example, under $rundir/core, the result for TEST001 (under
> >
> > > $scriptsdir/core) is LOG001.
> >
> > >
> >
> > > In my ~/.bashrc file, I have these two variables defined.
> >
> > >
> >
> > > export rundir=/mnt/qfc/rundir
> >
> > >
> >
> > > export scriptsdir=$trafodionRoot/sql/regress
> >
> > >
> >
> > >
> >
> > > After mkdir $rundir, make sure $rundir contains directories, one for
> >
> > > each category in $scriptsdir.  Once it is setup, you can run
> >
> > > regressions as many times as you want :-)
> >
> > >
> >
> > > On Tue, Aug 4, 2015 at 9:01 AM, Eric Owhadi <eric.owhadi@esgyn.com>
> > wrote:
> >
> > >
> >
> > > > That may be an obvious question, but I can’t find the logs and diffs
> >
> > > > for the first set of test showing up in the result of the runallsb:
> >
> > > >
> >
> > > > I can see all the logs and diffs for categories like core,
> >
> > > > compGeneral
> >
> > > etc,
> >
> > > > but the first set of test labelled:
> >
> > > >
> >
> > > > TEST001
> >
> > > >
> >
> > > > TEST002
> >
> > > >
> >
> > > > TEST003
> >
> > > >
> >
> > > > TEST004
> >
> > > >
> >
> > > > TEST005
> >
> > > >
> >
> > > > TEST006
> >
> > > >
> >
> > > > TEST015
> >
> > > >
> >
> > > > TEST017
> >
> > > >
> >
> > > > TEST018
> >
> > > >
> >
> > > > TEST020
> >
> > > >
> >
> > > >
> >
> > > >
> >
> > > > are not associated with a category, and they don’t have a
> >
> > > > subdirectory
> >
> > > with
> >
> > > > categoryname where I can find the other test results.
> >
> > > >
> >
> > > > Thanks in advance for the help.
> >
> > > >
> >
> > > >
> >
> > > >
> >
> > > > As a side note, if a test fails because of the compare, but the
> >
> > > > delta is legit, what should one do? Update known diffs? Or set a new
> >
> > > > compare file?
> >
> > > >
> >
> > > >
> >
> > > >
> >
> > > > Eric
> >
> > > >
> >
> > >
> >
> > >
> >
> > >
> >
> > > --
> >
> > > Regards, --Qifan
> >
> > >
> >
>

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message