trafodion-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Eric Owhadi <eric.owh...@esgyn.com>
Subject RE: Where are the logs for the first set of regression tests?
Date Tue, 04 Aug 2015 22:53:38 GMT
"regenerate a known diff file": is there a tool for that? Or is that a
manual process?
Eric

-----Original Message-----
From: Anoop Sharma [mailto:anoop.sharma@esgyn.com]
Sent: Tuesday, August 4, 2015 5:42 PM
To: dev@trafodion.incubator.apache.org
Subject: RE: Where are the logs for the first set of regression tests?

if a known diff file already exists for a test, then you cannot copy the log
file as the expected file. Doing that will mask the current known diffs.

You will need to manually update the expected file and regenerate a new
known diff file as diff line numbers may have changed.

anoop

-----Original Message-----
From: Eric Owhadi [mailto:eric.owhadi@esgyn.com]
Sent: Tuesday, August 4, 2015 3:37 PM
To: dev@trafodion.incubator.apache.org
Subject: Re: Where are the logs for the first set of regression tests?

OK clear,
Thanks again
Eric


On Tue, Aug 4, 2015 at 5:34 PM, Suresh Subbiah <suresh.subbiah60@gmail.com>
wrote:

> Hi Eric,
>
> Standard procedure is to modify the EXPECTEDaaa file. If you are
> satisfied with your new results you could simply copy the LOGaaa file
> as the new EXPECTEDaaa file. There are filters in the test suite that
> will hide things like timestamp from the final comparison.
> DIFFaaa.KNOWN files are meant to be temporary. To allow a test to pass
> while a fix is being worked on for a particular failure.
>
> Thanks
> Suresh
>
> On Tue, Aug 4, 2015 at 4:52 PM, Eric Owhadi <eric.owhadi@esgyn.com> wrote:
>
> > Thanks Sandhya, very useful!
> > BTW, if I want to change a test to take into account valid changes
> > that translate into false failure, should I modify the comparison
> > file, or the file that logs known legitimate differences?
> > Thanks again for the help :-),
> > Eric
> >
> > -----Original Message-----
> > From: Sandhya Sundaresan [mailto:sandhya.sundaresan@esgyn.com]
> > Sent: Tuesday, August 4, 2015 1:19 PM
> > To: dev@trafodion.incubator.apache.org
> > Subject: RE: Where are the logs for the first set of regression tests?
> >
> >  RE: Where are the logs for the first set of regression tests?
> >
> > There are logs and diffs. I notice I added the instructions on the
> > wiki only for the individual "pull request" automated jobs. I will
> > add them for the official test results as well.
> >
> > Here they are :
> >
> > Click on the link in the daily build email :
> >
> > *https://jenkins02.trafodion.org/job/Check-Daily/48/*
> > <https://jenkins02.trafodion.org/job/Check-Daily/48/>
> >
> > ==> Click on "Console Output" for any test suite you want to check
> >
> > ==> Around the 5th or 6th line you will see a link to detailed logs .
> Click
> > that.
> >
> >
> >
> >    2015-08-04 08:30:13 Started by upstream project "Check-Daily" build
> >       number 48
> >
> >       2015-08-04 08:30:13 originally caused by:
> >
> >       2015-08-04 08:30:13  Started by timer
> >
> >       2015-08-04 08:30:13 [EnvInject] - Loading node environment
> variables.
> >
> >       2015-08-04 08:30:13 Building remotely on ahw22-172.16.0.75
> >       (redhat_ahw2.2) in workspace
> >       /home/jenkins/workspace/core-regress-fullstack2-ahw2.2
> >
> >       2015-08-04 08:30:14 [core-regress-fullstack2-ahw2.2] $ /bin/sh
> >       /tmp/hudson2056470402644332566.sh
> >
> >       2015-08-04 08:30:14 Detailed logs:
> >       *
> >
> http://logs.trafodion.org/Daily/2015-08-04_08-30-00/regress-fullstack2
> -ahw2.2/*
> >       <
> >
> http://logs.trafodion.org/Daily/2015-08-04_08-30-00/regress-fullstack2
> -ahw2.2/
> > >
> >
> >         == > Navigate to sql-regress-log
> >
> >                 You will see a directory for the component  test
> > suite –
> eg
> > “executor” and a full  log – “Regress.log”.
> >
> >                 If you navigate to the component directory eg
> > executor,
> you
> > will see all the DIFFs and LOG files .
> >
> > Sandhya
> >
> > -----Original Message-----
> > From: Eric Owhadi [mailto:eric.owhadi@esgyn.com
> > <eric.owhadi@esgyn.com>]
> > Sent: Tuesday, August 4, 2015 11:06 AM
> > To: dev@trafodion.incubator.apache.org
> > Subject: RE: Where are the logs for the first set of regression tests?
> >
> > Hi Sandhya,
> >
> > Thanks for the offer, I'll first do a re-run with the feature off
> > and get familiar with the official results, that'll give me some
> > learning opportunity :-).
> >
> > There is no "diff" file on the official results automatic daily run?
> > So that it is easy to compare with the failure one get running the
> > test manually?
> >
> > Eric
> >
> > -----Original Message-----
> >
> > From: Sandhya Sundaresan [mailto:sandhya.sundaresan@esgyn.com
> > <sandhya.sundaresan@esgyn.com>]
> >
> > Sent: Tuesday, August 4, 2015 12:50 PM
> >
> > To: dev@trafodion.incubator.apache.org
> >
> > Subject: RE: Where are the logs for the first set of regression tests?
> >
> > Hi Eric,
> >
> >  They are run on a nightly basis. Attached are the results form last
> night
> > and previous night. As you can see 2 failures last night and both
> > are
> being
> > investigated. Otherwise we have been having good clean runs. So any
> > other failures you see need to be investigated. If you can give me
> > VNC access
> to
> > your machine, I can take a quick  look.
> >
> > I have documented how to read the official results on the wiki :
> >
> >
> >
> https://wiki.trafodion.org/wiki/index.php/Contributor_Workflow#Checkin
> g_Test_Results
> >
> > Sandhya
> >
> > -----Original Message-----
> >
> > From: Eric Owhadi [mailto:eric.owhadi@esgyn.com
> > <eric.owhadi@esgyn.com>]
> >
> > Sent: Tuesday, August 4, 2015 10:40 AM
> >
> > To: dev@trafodion.incubator.apache.org
> >
> > Subject: RE: Where are the logs for the first set of regression tests?
> >
> > Out of curiosity, anyone have recently run the regression test all
> > on master branch? I am getting some failure, and I wonder if they
> > are linked with
> my
> > changes or if the current regression suit is having known temporary
> issues?
> >
> > Eric
> >
> > -----Original Message-----
> >
> > From: Dave Birdsall [mailto:dave.birdsall@esgyn.com
> > <dave.birdsall@esgyn.com>]
> >
> > Sent: Tuesday, August 4, 2015 11:26 AM
> >
> > To: dev@trafodion.incubator.apache.org
> >
> > Subject: RE: Where are the logs for the first set of regression tests?
> >
> > Right. Start with the simple stuff I think. Eric's e-mail trail is
> valuable
> > as it is fresh newbie experience; if we could answer Eric's
> > questions on the wiki I think that would be a great start.
> >
> > -----Original Message-----
> >
> > From: Sandhya Sundaresan [mailto:sandhya.sundaresan@esgyn.com
> > <sandhya.sundaresan@esgyn.com>]
> >
> > Sent: Tuesday, August 4, 2015 9:24 AM
> >
> > To: dev@trafodion.incubator.apache.org
> >
> > Subject: RE: Where are the logs for the first set of regression tests?
> >
> > Hi Eric, Dave,
> >
> >   I can  add a page to the wiki for this.  We did have the steps
> documented
> > in our old internal wiki for those starting on Trafodion for the
> > first time.
> >
> > Also Roberta had a comprehensive  document describing the whole
> > regress environment back in MX days , I think. Not sure if anyone
> > has that saved.
> >
> > Either way, for the wiki, perhaps a smaller page would be sufficient.
> >
> > Thanks
> >
> > Sandhya
> >
> > -----Original Message-----
> >
> > From: Eric Owhadi [mailto:eric.owhadi@esgyn.com
> > <eric.owhadi@esgyn.com>]
> >
> > Sent: Tuesday, August 4, 2015 9:03 AM
> >
> > To: dev@trafodion.incubator.apache.org
> >
> > Subject: RE: Where are the logs for the first set of regression tests?
> >
> > I didn't want to ask, but indeed that would be fantastic and time
> > saving.
> >
> > Also how to write and/or modify when tests are complaining, but we
> > know that this is just because regression is based on delta, and
> > cannot recognize legitimate expected delta (due to know feature we
> > just implemented), and real regression.
> >
> > Eric
> >
> > -----Original Message-----
> >
> > From: Dave Birdsall [mailto:dave.birdsall@esgyn.com
> > <dave.birdsall@esgyn.com>]
> >
> > Sent: Tuesday, August 4, 2015 10:21 AM
> >
> > To: dev@trafodion.incubator.apache.org
> >
> > Subject: RE: Where are the logs for the first set of regression tests?
> >
> > Do we have a page on our Apache Trafodion wiki that describes how to
> > run the regression tests and how to interpret and deal with the
> > results? If
> someone
> > would like to volunteer to write such a page, it would be very
> > useful to newbies, I think.
> >
> > -----Original Message-----
> >
> > From: Eric Owhadi [mailto:eric.owhadi@esgyn.com
> > <eric.owhadi@esgyn.com>]
> >
> > Sent: Tuesday, August 4, 2015 7:45 AM
> >
> > To: dev@trafodion.incubator.apache.org
> >
> > Subject: RE: Where are the logs for the first set of regression tests?
> >
> > Thanks Suresh, that's it indeed :-). Was driving me crazy.
> >
> > Eric
> >
> > -----Original Message-----
> >
> > From: Suresh Subbiah [mailto:suresh.subbiah60@gmail.com
> > <suresh.subbiah60@gmail.com>]
> >
> > Sent: Tuesday, August 4, 2015 9:41 AM
> >
> > To: dev@trafodion.incubator.apache.org
> >
> > Subject: Re: Where are the logs for the first set of regression tests?
> >
> > Hi Eric,
> >
> > I think the "first" set of tests is the same as the last set. This
> > is an artifact of how runallsb prints out results. At the end of
> > every suite a summary of that suite is printed out (with no header
> > what suite it refers to since we have been running that suite for
> > the last X minutes). Then at
> the
> > end of a run it concatenates the summary of every suite, this time
> > with a header for each suite. So if you kick off regressions now and
> > come back several hours later there will be a screen with a summary
> > for all suites and then before the summary for core there will be
> > list of results with no suite name. This is the same as the last
> > suite in the summary (the hive test suite).
> >
> > Thanks
> >
> > Suresh
> >
> > On Tue, Aug 4, 2015 at 9:20 AM, Qifan Chen <qifan.chen@esgyn.com> wrote:
> >
> > > Hi Eric,
> >
> > >
> >
> > > The results should be in one of the directories under $rundir.
> > > For
> >
> > > example, under $rundir/core, the result for TEST001 (under
> >
> > > $scriptsdir/core) is LOG001.
> >
> > >
> >
> > > In my ~/.bashrc file, I have these two variables defined.
> >
> > >
> >
> > > export rundir=/mnt/qfc/rundir
> >
> > >
> >
> > > export scriptsdir=$trafodionRoot/sql/regress
> >
> > >
> >
> > >
> >
> > > After mkdir $rundir, make sure $rundir contains directories, one
> > > for
> >
> > > each category in $scriptsdir.  Once it is setup, you can run
> >
> > > regressions as many times as you want :-)
> >
> > >
> >
> > > On Tue, Aug 4, 2015 at 9:01 AM, Eric Owhadi
> > > <eric.owhadi@esgyn.com>
> > wrote:
> >
> > >
> >
> > > > That may be an obvious question, but I can’t find the logs and
> > > > diffs
> >
> > > > for the first set of test showing up in the result of the runallsb:
> >
> > > >
> >
> > > > I can see all the logs and diffs for categories like core,
> >
> > > > compGeneral
> >
> > > etc,
> >
> > > > but the first set of test labelled:
> >
> > > >
> >
> > > > TEST001
> >
> > > >
> >
> > > > TEST002
> >
> > > >
> >
> > > > TEST003
> >
> > > >
> >
> > > > TEST004
> >
> > > >
> >
> > > > TEST005
> >
> > > >
> >
> > > > TEST006
> >
> > > >
> >
> > > > TEST015
> >
> > > >
> >
> > > > TEST017
> >
> > > >
> >
> > > > TEST018
> >
> > > >
> >
> > > > TEST020
> >
> > > >
> >
> > > >
> >
> > > >
> >
> > > > are not associated with a category, and they don’t have a
> >
> > > > subdirectory
> >
> > > with
> >
> > > > categoryname where I can find the other test results.
> >
> > > >
> >
> > > > Thanks in advance for the help.
> >
> > > >
> >
> > > >
> >
> > > >
> >
> > > > As a side note, if a test fails because of the compare, but the
> >
> > > > delta is legit, what should one do? Update known diffs? Or set a
> > > > new
> >
> > > > compare file?
> >
> > > >
> >
> > > >
> >
> > > >
> >
> > > > Eric
> >
> > > >
> >
> > >
> >
> > >
> >
> > >
> >
> > > --
> >
> > > Regards, --Qifan
> >
> > >
> >
>

Mime
View raw message