trafodion-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Anoop Sharma <anoop.sha...@esgyn.com>
Subject RE: Where are the logs for the first set of regression tests?
Date Wed, 05 Aug 2015 00:55:23 GMT
no, there is no tool.

Once you get your DIFF file and it is different than the existing known diff
file, then you should validate what the diff of diffs are.

If the diff of diffs is only due to line numbers, then check in the new
diff file as the known diff file.

if diffs are not just line num diffs, then you need to validate  that this
new
set of diffs is expected due to your changes. If yes, then check in the diff
as
the known diff.

anoop

-----Original Message-----
From: Eric Owhadi [mailto:eric.owhadi@esgyn.com]
Sent: Tuesday, August 4, 2015 3:54 PM
To: dev@trafodion.incubator.apache.org
Subject: RE: Where are the logs for the first set of regression tests?

"regenerate a known diff file": is there a tool for that? Or is that a
manual process?
Eric

-----Original Message-----
From: Anoop Sharma [mailto:anoop.sharma@esgyn.com]
Sent: Tuesday, August 4, 2015 5:42 PM
To: dev@trafodion.incubator.apache.org
Subject: RE: Where are the logs for the first set of regression tests?

if a known diff file already exists for a test, then you cannot copy the log
file as the expected file. Doing that will mask the current known diffs.

You will need to manually update the expected file and regenerate a new
known diff file as diff line numbers may have changed.

anoop

-----Original Message-----
From: Eric Owhadi [mailto:eric.owhadi@esgyn.com]
Sent: Tuesday, August 4, 2015 3:37 PM
To: dev@trafodion.incubator.apache.org
Subject: Re: Where are the logs for the first set of regression tests?

OK clear,
Thanks again
Eric


On Tue, Aug 4, 2015 at 5:34 PM, Suresh Subbiah <suresh.subbiah60@gmail.com>
wrote:

> Hi Eric,
>
> Standard procedure is to modify the EXPECTEDaaa file. If you are
> satisfied with your new results you could simply copy the LOGaaa file
> as the new EXPECTEDaaa file. There are filters in the test suite that
> will hide things like timestamp from the final comparison.
> DIFFaaa.KNOWN files are meant to be temporary. To allow a test to pass
> while a fix is being worked on for a particular failure.
>
> Thanks
> Suresh
>
> On Tue, Aug 4, 2015 at 4:52 PM, Eric Owhadi <eric.owhadi@esgyn.com> wrote:
>
> > Thanks Sandhya, very useful!
> > BTW, if I want to change a test to take into account valid changes
> > that translate into false failure, should I modify the comparison
> > file, or the file that logs known legitimate differences?
> > Thanks again for the help :-),
> > Eric
> >
> > -----Original Message-----
> > From: Sandhya Sundaresan [mailto:sandhya.sundaresan@esgyn.com]
> > Sent: Tuesday, August 4, 2015 1:19 PM
> > To: dev@trafodion.incubator.apache.org
> > Subject: RE: Where are the logs for the first set of regression tests?
> >
> >  RE: Where are the logs for the first set of regression tests?
> >
> > There are logs and diffs. I notice I added the instructions on the
> > wiki only for the individual "pull request" automated jobs. I will
> > add them for the official test results as well.
> >
> > Here they are :
> >
> > Click on the link in the daily build email :
> >
> > *https://jenkins02.trafodion.org/job/Check-Daily/48/*
> > <https://jenkins02.trafodion.org/job/Check-Daily/48/>
> >
> > ==> Click on "Console Output" for any test suite you want to check
> >
> > ==> Around the 5th or 6th line you will see a link to detailed logs .
> Click
> > that.
> >
> >
> >
> >    2015-08-04 08:30:13 Started by upstream project "Check-Daily" build
> >       number 48
> >
> >       2015-08-04 08:30:13 originally caused by:
> >
> >       2015-08-04 08:30:13  Started by timer
> >
> >       2015-08-04 08:30:13 [EnvInject] - Loading node environment
> variables.
> >
> >       2015-08-04 08:30:13 Building remotely on ahw22-172.16.0.75
> >       (redhat_ahw2.2) in workspace
> >       /home/jenkins/workspace/core-regress-fullstack2-ahw2.2
> >
> >       2015-08-04 08:30:14 [core-regress-fullstack2-ahw2.2] $ /bin/sh
> >       /tmp/hudson2056470402644332566.sh
> >
> >       2015-08-04 08:30:14 Detailed logs:
> >       *
> >
> http://logs.trafodion.org/Daily/2015-08-04_08-30-00/regress-fullstack2
> -ahw2.2/*
> >       <
> >
> http://logs.trafodion.org/Daily/2015-08-04_08-30-00/regress-fullstack2
> -ahw2.2/
> > >
> >
> >         == > Navigate to sql-regress-log
> >
> >                 You will see a directory for the component  test
> > suite –
> eg
> > “executor” and a full  log – “Regress.log”.
> >
> >                 If you navigate to the component directory eg
> > executor,
> you
> > will see all the DIFFs and LOG files .
> >
> > Sandhya
> >
> > -----Original Message-----
> > From: Eric Owhadi [mailto:eric.owhadi@esgyn.com
> > <eric.owhadi@esgyn.com>]
> > Sent: Tuesday, August 4, 2015 11:06 AM
> > To: dev@trafodion.incubator.apache.org
> > Subject: RE: Where are the logs for the first set of regression tests?
> >
> > Hi Sandhya,
> >
> > Thanks for the offer, I'll first do a re-run with the feature off
> > and get familiar with the official results, that'll give me some
> > learning opportunity :-).
> >
> > There is no "diff" file on the official results automatic daily run?
> > So that it is easy to compare with the failure one get running the
> > test manually?
> >
> > Eric
> >
> > -----Original Message-----
> >
> > From: Sandhya Sundaresan [mailto:sandhya.sundaresan@esgyn.com
> > <sandhya.sundaresan@esgyn.com>]
> >
> > Sent: Tuesday, August 4, 2015 12:50 PM
> >
> > To: dev@trafodion.incubator.apache.org
> >
> > Subject: RE: Where are the logs for the first set of regression tests?
> >
> > Hi Eric,
> >
> >  They are run on a nightly basis. Attached are the results form last
> night
> > and previous night. As you can see 2 failures last night and both
> > are
> being
> > investigated. Otherwise we have been having good clean runs. So any
> > other failures you see need to be investigated. If you can give me
> > VNC access
> to
> > your machine, I can take a quick  look.
> >
> > I have documented how to read the official results on the wiki :
> >
> >
> >
> https://wiki.trafodion.org/wiki/index.php/Contributor_Workflow#Checkin
> g_Test_Results
> >
> > Sandhya
> >
> > -----Original Message-----
> >
> > From: Eric Owhadi [mailto:eric.owhadi@esgyn.com
> > <eric.owhadi@esgyn.com>]
> >
> > Sent: Tuesday, August 4, 2015 10:40 AM
> >
> > To: dev@trafodion.incubator.apache.org
> >
> > Subject: RE: Where are the logs for the first set of regression tests?
> >
> > Out of curiosity, anyone have recently run the regression test all
> > on master branch? I am getting some failure, and I wonder if they
> > are linked with
> my
> > changes or if the current regression suit is having known temporary
> issues?
> >
> > Eric
> >
> > -----Original Message-----
> >
> > From: Dave Birdsall [mailto:dave.birdsall@esgyn.com
> > <dave.birdsall@esgyn.com>]
> >
> > Sent: Tuesday, August 4, 2015 11:26 AM
> >
> > To: dev@trafodion.incubator.apache.org
> >
> > Subject: RE: Where are the logs for the first set of regression tests?
> >
> > Right. Start with the simple stuff I think. Eric's e-mail trail is
> valuable
> > as it is fresh newbie experience; if we could answer Eric's
> > questions on the wiki I think that would be a great start.
> >
> > -----Original Message-----
> >
> > From: Sandhya Sundaresan [mailto:sandhya.sundaresan@esgyn.com
> > <sandhya.sundaresan@esgyn.com>]
> >
> > Sent: Tuesday, August 4, 2015 9:24 AM
> >
> > To: dev@trafodion.incubator.apache.org
> >
> > Subject: RE: Where are the logs for the first set of regression tests?
> >
> > Hi Eric, Dave,
> >
> >   I can  add a page to the wiki for this.  We did have the steps
> documented
> > in our old internal wiki for those starting on Trafodion for the
> > first time.
> >
> > Also Roberta had a comprehensive  document describing the whole
> > regress environment back in MX days , I think. Not sure if anyone
> > has that saved.
> >
> > Either way, for the wiki, perhaps a smaller page would be sufficient.
> >
> > Thanks
> >
> > Sandhya
> >
> > -----Original Message-----
> >
> > From: Eric Owhadi [mailto:eric.owhadi@esgyn.com
> > <eric.owhadi@esgyn.com>]
> >
> > Sent: Tuesday, August 4, 2015 9:03 AM
> >
> > To: dev@trafodion.incubator.apache.org
> >
> > Subject: RE: Where are the logs for the first set of regression tests?
> >
> > I didn't want to ask, but indeed that would be fantastic and time
> > saving.
> >
> > Also how to write and/or modify when tests are complaining, but we
> > know that this is just because regression is based on delta, and
> > cannot recognize legitimate expected delta (due to know feature we
> > just implemented), and real regression.
> >
> > Eric
> >
> > -----Original Message-----
> >
> > From: Dave Birdsall [mailto:dave.birdsall@esgyn.com
> > <dave.birdsall@esgyn.com>]
> >
> > Sent: Tuesday, August 4, 2015 10:21 AM
> >
> > To: dev@trafodion.incubator.apache.org
> >
> > Subject: RE: Where are the logs for the first set of regression tests?
> >
> > Do we have a page on our Apache Trafodion wiki that describes how to
> > run the regression tests and how to interpret and deal with the
> > results? If
> someone
> > would like to volunteer to write such a page, it would be very
> > useful to newbies, I think.
> >
> > -----Original Message-----
> >
> > From: Eric Owhadi [mailto:eric.owhadi@esgyn.com
> > <eric.owhadi@esgyn.com>]
> >
> > Sent: Tuesday, August 4, 2015 7:45 AM
> >
> > To: dev@trafodion.incubator.apache.org
> >
> > Subject: RE: Where are the logs for the first set of regression tests?
> >
> > Thanks Suresh, that's it indeed :-). Was driving me crazy.
> >
> > Eric
> >
> > -----Original Message-----
> >
> > From: Suresh Subbiah [mailto:suresh.subbiah60@gmail.com
> > <suresh.subbiah60@gmail.com>]
> >
> > Sent: Tuesday, August 4, 2015 9:41 AM
> >
> > To: dev@trafodion.incubator.apache.org
> >
> > Subject: Re: Where are the logs for the first set of regression tests?
> >
> > Hi Eric,
> >
> > I think the "first" set of tests is the same as the last set. This
> > is an artifact of how runallsb prints out results. At the end of
> > every suite a summary of that suite is printed out (with no header
> > what suite it refers to since we have been running that suite for
> > the last X minutes). Then at
> the
> > end of a run it concatenates the summary of every suite, this time
> > with a header for each suite. So if you kick off regressions now and
> > come back several hours later there will be a screen with a summary
> > for all suites and then before the summary for core there will be
> > list of results with no suite name. This is the same as the last
> > suite in the summary (the hive test suite).
> >
> > Thanks
> >
> > Suresh
> >
> > On Tue, Aug 4, 2015 at 9:20 AM, Qifan Chen <qifan.chen@esgyn.com> wrote:
> >
> > > Hi Eric,
> >
> > >
> >
> > > The results should be in one of the directories under $rundir.
> > > For
> >
> > > example, under $rundir/core, the result for TEST001 (under
> >
> > > $scriptsdir/core) is LOG001.
> >
> > >
> >
> > > In my ~/.bashrc file, I have these two variables defined.
> >
> > >
> >
> > > export rundir=/mnt/qfc/rundir
> >
> > >
> >
> > > export scriptsdir=$trafodionRoot/sql/regress
> >
> > >
> >
> > >
> >
> > > After mkdir $rundir, make sure $rundir contains directories, one
> > > for
> >
> > > each category in $scriptsdir.  Once it is setup, you can run
> >
> > > regressions as many times as you want :-)
> >
> > >
> >
> > > On Tue, Aug 4, 2015 at 9:01 AM, Eric Owhadi
> > > <eric.owhadi@esgyn.com>
> > wrote:
> >
> > >
> >
> > > > That may be an obvious question, but I can’t find the logs and
> > > > diffs
> >
> > > > for the first set of test showing up in the result of the runallsb:
> >
> > > >
> >
> > > > I can see all the logs and diffs for categories like core,
> >
> > > > compGeneral
> >
> > > etc,
> >
> > > > but the first set of test labelled:
> >
> > > >
> >
> > > > TEST001
> >
> > > >
> >
> > > > TEST002
> >
> > > >
> >
> > > > TEST003
> >
> > > >
> >
> > > > TEST004
> >
> > > >
> >
> > > > TEST005
> >
> > > >
> >
> > > > TEST006
> >
> > > >
> >
> > > > TEST015
> >
> > > >
> >
> > > > TEST017
> >
> > > >
> >
> > > > TEST018
> >
> > > >
> >
> > > > TEST020
> >
> > > >
> >
> > > >
> >
> > > >
> >
> > > > are not associated with a category, and they don’t have a
> >
> > > > subdirectory
> >
> > > with
> >
> > > > categoryname where I can find the other test results.
> >
> > > >
> >
> > > > Thanks in advance for the help.
> >
> > > >
> >
> > > >
> >
> > > >
> >
> > > > As a side note, if a test fails because of the compare, but the
> >
> > > > delta is legit, what should one do? Update known diffs? Or set a
> > > > new
> >
> > > > compare file?
> >
> > > >
> >
> > > >
> >
> > > >
> >
> > > > Eric
> >
> > > >
> >
> > >
> >
> > >
> >
> > >
> >
> > > --
> >
> > > Regards, --Qifan
> >
> > >
> >
>

Mime
View raw message