trafodion-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Gunnar Tapper <tapper.gun...@gmail.com>
Subject Re: Parallel Make Failures
Date Tue, 08 Mar 2016 02:20:38 GMT
My problems is that the include and lib directories are created
under ~/trafodion-build-tools/thrift-0.9.0/. So, I'd be pointing these
variables to non-existing directories. The effect is the same:

[centos@gtdev incubator-trafodion]$ make all -j 1
Building all Trafodion components
cd core && make all
make[1]: Entering directory `/home/centos/incubator-trafodion/core'
./bldenvchk.sh;
*** Warning: QT_TOOLKIT does not point to an existing directory.
*** Warning: SQL Compiler Debugger will not be built.
*** Error: THRIFT_LIB_DIR directory not found:
/home/centos/trafodion-build-tools/thrift-0.9.0/lib
*** Error: THRIFT_INC_DIR directory not found:
/home/centos/trafodion-build-tools/thrift-0.9.0/include
make[1]: *** [sqroot] Error 1
make[1]: Leaving directory `/home/centos/incubator-trafodion/core'
make: *** [all] Error 2
[centos@gtdev incubator-trafodion]$ env | grep THRIFT
THRIFT_LIB_DIR=/home/centos/trafodion-build-tools/thrift-0.9.0/lib
THRIFT_INC_DIR=/home/centos/trafodion-build-tools/thrift-0.9.0/include
THRIFT_DEP_VER=0.9.0


On Mon, Mar 7, 2016 at 7:12 PM, Roberta Marton <roberta.marton@esgyn.com>
wrote:

> I was wondering if you did setup them up before building if it fixed the
> build issue.
>
>     Roberta
>
> -----Original Message-----
> From: Gunnar Tapper [mailto:tapper.gunnar@gmail.com]
> Sent: Monday, March 7, 2016 6:07 PM
> To: dev@trafodion.incubator.apache.org
> Subject: Re: Parallel Make Failures
>
> Do I set these variables before running the tools setup?
>
> On Mon, Mar 7, 2016 at 6:55 PM, Roberta Marton <roberta.marton@esgyn.com>
> wrote:
>
> > When I was testing release 1.3 last year, I remember getting a similar
> > issue.  Thrift was installed but something in the make was not working
> > correctly.
> > To get around the problem temporarily, I manually exported the
> > THRIFT_INC_DIR and THRIFT_LIB_DIR:
> >
> > export THRIFT_LIB_DIR=<location>
> > export THRIFT_INC_DIR=<location>
> >
> > The build then succeeded.
> > There were several later changes made in the setup files that
> > eventually did not require this workaround.
> >
> > If you manually set these envvars, does the build work?
> >
> >      Roberta
> >
> > -----Original Message-----
> > From: Gunnar Tapper [mailto:tapper.gunnar@gmail.com]
> > Sent: Monday, March 7, 2016 5:28 PM
> > To: dev@trafodion.incubator.apache.org
> > Subject: Re: Parallel Make Failures
> >
> > I created a new VM and did a new git clone. As stated, include and lib
> > are not built for Thrift. Also, it seems to me that the qmake change
> > might be broken. See below.
> >
> > [centos@trafbuild ~]$ cd incubator-trafodion/ [centos@trafbuild
> > incubator-trafodion]$ source ./env.sh
> >
> >     If you are ready to build Trafodion, perform one of the following
> > options:
> >
> >       make all         (Build Trafodion, DCS, and REST) OR
> >       make package     (Build Trafodion, DCS, REST, and Client drivers)
> > OR
> >       make package-all (Build Trafodion, DCS, REST, Client drivers,
> > and
> > Tests)
> >
> >     If Trafodion has been built and you want test:
> >
> >        Execute the install_local_hadoop script which performs a single
> > node
> >        install using a popular Hadoop distribution
> >
> >           cd /home/centos/incubator-trafodion/core/sqf/sql/scripts
> >           install_local_hadoop [-p <port option>]
> >           install_traf_components
> >           configure Trafodion and start the processes
> >           HAVE FUN!
> >
> >    You can also choose to install_local_hadoop before building Trafodion.
> >
> > [centos@trafbuild incubator-trafodion]$ make all -j 1 Building all
> > Trafodion components cd core && make all
> > make[1]: Entering directory `/home/centos/incubator-trafodion/core'
> > ./bldenvchk.sh;
> > *** Warning: QT_TOOLKIT does not point to an existing directory.
> > *** Warning: SQL Compiler Debugger will not be built.
> > *** Error: THRIFT_LIB_DIR directory not found:
> > //home/centos/trafodion-build-tools/thrift-0.9.0/lib
> > *** Error: THRIFT_INC_DIR directory not found:
> > //home/centos/trafodion-build-tools/thrift-0.9.0/include
> > make[1]: *** [sqroot] Error 1
> > make[1]: Leaving directory `/home/centos/incubator-trafodion/core'
> > make: *** [all] Error 2
> >
> > [centos@trafbuild incubator-trafodion]$ echo $QT_TOOLKIT
> >
> > [centos@trafbuild incubator-trafodion]$ ls
> > ~/trafodion-build-tools/thrift-0.9.0/
> > bin
> >
> >
> > On Mon, Mar 7, 2016 at 3:16 PM, Steve Varnau <steve.varnau@esgyn.com>
> > wrote:
> >
> > > The traf_tools_setup.sh script has not changed in the in the last
> > > few days.
> > > I hope that the download location for thrift have not changed.  The
> > > only thing I can see that has been changing is the wiki instructions
> > > for setting up the environment.
> > >
> > > I see at least one thing wrong on wiki. For instance, it says that
> > > CentOS
> > > 6.7 won't work.  That is not correct. There was a mistaken
> > > impression that
> > > 6.7 has newer gcc. Not true.
> > >
> > > With the very latest code (as of this morning), the qt-devel and
> > > qt-config should no longer be needed.
> > >
> > > --Steve
> > >
> > >
> > > > -----Original Message-----
> > > > From: Gunnar Tapper [mailto:tapper.gunnar@gmail.com]
> > > > Sent: Monday, March 7, 2016 2:03 PM
> > > > To: dev@trafodion.incubator.apache.org
> > > > Subject: Re: Parallel Make Failures
> > > >
> > > > Alas, things are getting worse rather than better... :(
> > > >
> > > > I create a new VM, installed all required packages, did a git
> > > > clone of trafodion, and then I ran install/traf_tools_setup.sh.
> > > > Now, Thrift isn't building the include and lib folders. I run the
> > > > exact same command on the version that I downloaded last Thursday
> > > > and that works just fine.
> > > >
> > > > Did anything change that I should be aware of?
> > > >
> > > > Gunnar
> > > >
> > > > On Mon, Mar 7, 2016 at 12:17 PM, Gunnar Tapper
> > > > <tapper.gunnar@gmail.com>
> > > > wrote:
> > > >
> > > > > Well, today, the build doesn't work at all regardless of what -j
> > > > > or -l options I use. It constantly fails on:
> > > > >
> > > > > Generating C++ code from yacc file ../sqlci/sqlci_yacc.y
> > > > > ##(SQL)
> > > > > ../sqlci/sqlci_yacc.y: warning: 1 reduce/reduce conflict
> > > > > [-Wconflicts-rr]
> > > > >       ##(SQL)
> > > > > /home/centos/traf-tools/bison_3_linux/bin/bison:
> > > > >
> /home/centos/trafodion//bison_3_linux/share/bison/m4sugar/m4sugar.m4:
> > > > > cannot open: No such file or directory   ##(SQL)
> > > > > mv: cannot stat `sqlcilib/linux/64bit/debug/sqlci_yacc.cpp': No
> > > > > such file or directory  ##(SQL)
> > > > > sed: can't read sqlcilib/linux/64bit/debug/sqlci_yacc.cpp.tmp:
> > > > > No
> > such
> > > > > file or directory        ##(SQL)
> > > > > sed: can't read sqlcilib/linux/64bit/debug/sqlci_yacc.hpp: No
> > > > > such file or
> > > > > directory    ##(SQL)
> > > > > rm: cannot remove `sqlcilib/linux/64bit/debug/sqlci_yacc.hpp':
> > > > > No
> > such
> > > > > file or directory        ##(SQL)
> > > > > rm: cannot remove `sqlcilib/linux/64bit/debug/sqlci_yacc.cpp.tmp':
> > > > > No such
> > > > > file or directory    ##(SQL)
> > > > > make[4]: *** [sqlcilib/linux/64bit/debug/sqlci_yacc.h] Error 1
> > > > > ##(SQL)
> > > > > make[4]: Leaving directory
> > > > > `/home/centos/incubator-trafodion/core/sql/nskgmake' ##(SQL)
> > > > > make[3]: *** [all] Error 2      ##(SQL)
> > > > > make[3]: Leaving directory
> > > > > `/home/centos/incubator-trafodion/core/sqf/sql'
> > > > >      ##(SQL)
> > > > > make[2]: *** [make_sql] Error 2
> > > > > make[2]: Leaving directory
> > `/home/centos/incubator-trafodion/core/sqf'
> > > > > make[1]: *** [foundation] Error 2
> > > > > make[1]: Leaving directory `/home/centos/incubator-trafodion/core'
> > > > > make: *** [all] Error 2
> > > > >
> > > > >
> > > > > On Mon, Mar 7, 2016 at 12:05 PM, Amanda Moran
> > > > <amanda.moran@esgyn.com>
> > > > > wrote:
> > > > >
> > > > >> On redhat 7.1 this command still works:
> > > > >>
> > > > >> [ec2-user@ip-10-0-0-175 ~]$ cat /etc/redhat-release Red Hat
> > > > >> Enterprise Linux Server release 7.1 (Maipo)
> > > > >>
> > > > >> [ec2-user@ip-10-0-0-175 ~]$ grep processor /proc/cpuinfo | wc
> > > > >> -l
> > > > >> 4
> > > > >>
> > > > >>
> > > > >> On Mon, Mar 7, 2016 at 9:56 AM, Steve Varnau
> > > > >> <steve.varnau@esgyn.com>
> > > > >> wrote:
> > > > >>
> > > > >> > > It seems that the parallel make fails on 8 GB machines.
> > > > >> >
> > > > >> > I think your first sentence overstates the determinism of
the
> > > problem
> > > > >> > a
> > > > >> > bit.
> > > > >> > I ran a normal, default build on 8GB machine last week and
> > > > >> > had no
> > > > >> problem.
> > > > >> > There must be an environmental problem, but I don't think
we
> > > > >> > fully understand it yet.
> > > > >> >
> > > > >> > The aggressiveness of the make parallelism is set in
> > > > >> core/sqf/sqenvcom.sh.
> > > > >> > It sets the parallel factor based on how many CPUs are on
> > > > >> > your
> > > > >> > machine:
> > > > >> >
> > > > >> > # Set default build parallelism # Can be overridden on make
> > > > >> > commandline cpucnt=$(grep processor /proc/cpuinfo | wc -l)
> > > > >> > #     no number means unlimited, and will swamp the system
> > > > >> > export MAKEFLAGS="-j$cpucnt"
> > > > >> >
> > > > >> > If that calculation is wrong, maybe that could cause a problem.
> > > > >> >
> > > > >> > --Steve
> > > > >> >
> > > > >> >
> > > > >> > > -----Original Message-----
> > > > >> > > From: Gunnar Tapper [mailto:tapper.gunnar@gmail.com]
> > > > >> > > Sent: Monday, March 7, 2016 9:35 AM
> > > > >> > > To: dev@trafodion.incubator.apache.org
> > > > >> > > Subject: Parallel Make Failures
> > > > >> > >
> > > > >> > > Hi,
> > > > >> > >
> > > > >> > > It seems that the parallel make fails on 8 GB machines.
At
> > > > >> > > least,
> > > > >> Nitin
> > > > >> > > and
> > > > >> > > I both ran into make failures that did not appear when
> > > > >> > > running serial make.
> > > > >> > > I've also seen similar failures when building the code
on
> > > > >> > > 12 GB
> > > > >> machines.
> > > > >> > >
> > > > >> > > Based on previous discussions, the Trafodion Contributor
> > > > >> > > Guide
> > > > >> recommends
> > > > >> > > rerunning make a few times if running issues.
> > > > >> > >
> > > > >> > > I most wonder if there's a way to reduce the aggressiveness
> > > > >> > > of the
> > > > >> make
> > > > >> > in
> > > > >> > > general. Could we, for example, come up with a table
that
> > > > >> > > correlates system size to define the -l option or something
> > > > >> > > similar?
> > > > >> > >
> > > > >> > > --
> > > > >> > > Thanks,
> > > > >> > >
> > > > >> > > Gunnar
> > > > >> > > *If you think you can you can, if you think you can't
> > > > >> > > you're
> > > > >> > > right.*
> > > > >> >
> > > > >>
> > > > >>
> > > > >>
> > > > >> --
> > > > >> Thanks,
> > > > >>
> > > > >> Amanda Moran
> > > > >>
> > > > >
> > > > >
> > > > >
> > > > > --
> > > > > Thanks,
> > > > >
> > > > > Gunnar
> > > > > *If you think you can you can, if you think you can't you're
> > > > > right.*
> > > > >
> > > >
> > > >
> > > >
> > > > --
> > > > Thanks,
> > > >
> > > > Gunnar
> > > > *If you think you can you can, if you think you can't you're
> > > > right.*
> > >
> >
> >
> >
> > --
> > Thanks,
> >
> > Gunnar
> > *If you think you can you can, if you think you can't you're right.*
> >
>
>
>
> --
> Thanks,
>
> Gunnar
> *If you think you can you can, if you think you can't you're right.*
>



-- 
Thanks,

Gunnar
*If you think you can you can, if you think you can't you're right.*

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message