trafodion-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Radu Marias <radumar...@gmail.com>
Subject Re: HammerDB issues
Date Fri, 11 Sep 2015 19:25:36 GMT
Dave, Narendra

BTW, Amanda has access to our cluster. Please feel free to take a look if
you think it helps in debugging the issue.

On Fri, Sep 11, 2015 at 10:24 PM, Radu Marias <radumarias@gmail.com> wrote:

> $ file core.36479
> core.36479: ELF 64-bit LSB core file x86-64, version 1 (SYSV), SVR4-style,
> from 'tdm_arkesp SQMON1.1 00000 00000 036479 $Z000US9 <IP>:35596 00004 0000'
>
>
> On Fri, Sep 11, 2015 at 10:22 PM, Narendra Goyal <narendra.goyal@esgyn.com
> > wrote:
>
>> Hi Radu,
>>
>> Could you please run 'file core*' in the same directory. At least it will
>> identify the program that core'd.
>>
>> Thanks,
>> -Narendra
>>
>> -----Original Message-----
>> From: Radu Marias [mailto:radumarias@gmail.com]
>> Sent: Friday, September 11, 2015 12:15 PM
>> To: dev <dev@trafodion.incubator.apache.org>
>> Subject: Re: HammerDB issues
>>
>> yes,
>>
>> $ pwd
>> /home/trafodion/trafodion-20150828_0830/sql/scripts
>> $ ls -ltrh | grep core.
>> -rwxr-x--- 1 trafodion trafodion 3.3K Aug 28 08:30 sqcorefile
>> -rw------- 1 trafodion trafodion 152M Sep 11 13:27 core.42133
>> -rw------- 1 trafodion trafodion 162M Sep 11 17:36 core.9560
>> -rw------- 1 trafodion trafodion 162M Sep 11 17:39 core.10408
>> -rw------- 1 trafodion trafodion 152M Sep 11 18:27 core.36479
>>
>> All are about 5.3 MB archived. Do you want to connect to the cluster and
>> debug them? Or should I share over email? Because I don't know if
>> sensitive
>> information is included to post them to mailing list also.
>>
>> A quick look, but don't think it helps without symbol-file:
>>
>> Program terminated with signal 11, Segmentation fault.
>> #0  0x00007f619ae22b56 in ?? ()
>> (gdb) bt
>> #0  0x00007f619ae22b56 in ?? ()
>> #1  0x00007f619ae0d710 in ?? ()
>> #2  0x00007f618ceafb30 in ?? ()
>> #3  0x00007fffe7279c10 in ?? ()
>> #4  0x000000000040bd4d in ?? ()
>> #5  0x00007f618ceafb30 in ?? ()
>> #6  0x0000000000000000 in ?? ()
>>
>> On Fri, Sep 11, 2015 at 9:53 PM, Dave Birdsall <dave.birdsall@esgyn.com>
>> wrote:
>>
>> > Sorry, I meant core dumps from abending processes.
>> >
>> > See any files with file name "core.<node name>.<pid>.<executable
>> > name>" out there?
>> >
>> > If so might be interesting to use gdb to see the stack trace.
>> >
>> > -----Original Message-----
>> > From: Radu Marias [mailto:radumarias@gmail.com]
>> > Sent: Friday, September 11, 2015 11:44 AM
>> > To: dev <dev@trafodion.incubator.apache.org>
>> > Subject: Re: HammerDB issues
>> >
>> > By "core files" you mean mxssmp processes? If so then this is present
>> > on all
>> > nodes:
>> >
>> > $ ps x | grep mxssmp
>> > 46157 ?        SNl    0:02 mxssmp SQMON1.1 00004 00004 046157 $ZSM004
>> > 188.138.61.180:35762 00011 00004 00005 SPARE
>> >
>> > I tried this:
>> >
>> > sqstop (then needed to use ckillall because it was stuck) restarted
>> > hdp sqstart
>> >
>> > Now this works from sqlci:
>> >
>> > CREATE UNIQUE INDEX CUSTOMER_I100 ON CUSTOMER (C_W_ID, C_D_ID, C_LAST,
>> > C_FIRST, C_ID);
>> >
>> > But when I run hammerdb without SPJ it inserts data in some tables and
>> > then it fails again with
>> >
>> > Vuser 1:Creating Index CUSTOMER_I2...
>> > Vuser 1:Failed to Create Index
>> >
>> > And now if I run the create statement in sqlci I get the same error
>> > "Operating system error 201".
>> >
>> > *So it seems that while hammerdb inserts data some trafodion processes
>> > went away unexpectedly?*
>> >
>> > On Fri, Sep 11, 2015 at 8:47 PM, Dave Birdsall
>> > <dave.birdsall@esgyn.com>
>> > wrote:
>> >
>> > > The first one ("Operating system error 201") means a process went
>> > > away unexpectedly. Do you see any core files?
>> > >
>> > > -----Original Message-----
>> > > From: Radu Marias [mailto:radumarias@gmail.com]
>> > > Sent: Friday, September 11, 2015 10:46 AM
>> > > To: dev <dev@trafodion.incubator.apache.org>
>> > > Subject: Re: HammerDB issues
>> > >
>> > > This is what I get for the DDL you mentioned:
>> > >
>> > > >>CREATE UNIQUE INDEX CUSTOMER_I2 ON CUSTOMER (C_W_ID, C_D_ID,
>> > > >>C_LAST,
>> > > C_FIRST, C_ID);
>> > >
>> > > *** ERROR[2034] $Z00083Z:144: Operating system error 201 while
>> > > communicating with server process $Z0201A7:26.
>> > >
>> > > *** ERROR[1081] Loading of index TRAFODION.TPCC.CUSTOMER_I2 failed
>> > > unexpectedly.
>> > >
>> > > --- SQL operation failed with errors.
>> > >
>> > > The one for ORDERS is working:
>> > >
>> > > >>CREATE UNIQUE INDEX ORDERS_I2 ON ORDERS (O_W_ID, O_D_ID, O_C_ID,
>> > > >>O_ID);
>> > >
>> > > --- SQL operation complete.
>> > >
>> > > $ sqcheck
>> > > Checking if processes are up.
>> > > Checking attempt: 1; user specified max: 2. Execution time in seconds:
>> > > 0.
>> > >
>> > > The SQ environment is up!
>> > >
>> > >
>> > > Process         Configured      Actual      Down
>> > > -------         ----------      ------      ----
>> > > DTM             5               5
>> > > RMS             10              10
>> > > MXOSRVR         20              20
>> > >
>> > > Don't see any errors in trafodion logs.
>> > >
>> > > On Fri, Sep 11, 2015 at 8:19 PM, Radu Marias <radumarias@gmail.com>
>> > wrote:
>> > >
>> > > > pdsh is available but how should the jar files be replicated?
>> > > >
>> > > > I also installed HammerDB on the other 4 nodes and setup first
>> > > > node as master and the 4 nodes as slaves as described here
>> > > > http://hammerora.sourceforge.net/hammerdb_remote_modes.pdf
>> > > > I also checked "*Copy Stored Procedures to Remote Nodes*" and
>> > > > added the list of all 5 nodes but I don't see *NEWORDER.jar* on
>> > > > the other
>> > > > 4 nodes in *HammerDB-2.18* folder. I assume all the jars are
>> > > > generated on the first node and copied to the others. But it only
>> > > > generates
>> > > > *NEWORDER.jar* on the first node and it fails and it's not present
>> > > > on the other nodes. Also the rest of the stored procedure jars are
>> > > > not created.
>> > > >
>> > > > For the create index DDL used, I changed config.xml file from
>> > > > hammerdb to enable logs in /tmp and in that file I can only see
>> > > > the trimmed statement.
>> > > > Is there anywhere I can see additional logs? Or can I see in
>> > > > trafodion logs the last statement executed from hammerdb over odbc?
>> > > >
>> > > > On Fri, Sep 11, 2015 at 6:38 PM, Suresh Subbiah <
>> > > > suresh.subbiah60@gmail.com> wrote:
>> > > >
>> > > >> Hi Radu,
>> > > >>
>> > > >> Thanks for trying this out.
>> > > >>
>> > > >> I think what is happening here is that the jar file did not get
>> > > >> pushed to all the nodes. HammerDB has so far been tested on
>> > > >> single node instances or multiple nodes where we did some
>> > > >> intermediate steps manually. I don't think HammerDB uses pdsh
to
>> > > >> push the jar file to all nodes, or maybe pdsh is somehow not
>> > > >> available. Could you please check if the jar file is available
>> > > >> from every node in the cluster? If this does turn out to be the
>> > > >> problem, I promise to add more info about HammerDB in the
>> > > >> Trafodion wiki and mention a workaround for these issues.
>> > > >>
>> > > >> For the Failed To Create Index issue, is there an error message?
>> > > >> IF not we can take the create index statement HammerDB is using
>> > > >> and issue it from sqlci. I think thisis the DDL that is used.
>> > > >>
>> > > >> CREATE UNIQUE INDEX CUSTOMER_I2 ON CUSTOMER (C_W_ID, C_D_ID,
>> > > >> C_LAST, C_FIRST, C_ID) ; CREATE UNIQUE INDEX ORDERS_I2 ON ORDERS
>> > > >> (O_W_ID, O_D_ID, O_C_ID, O_ID) ;
>> > > >>
>> > > >> Thanks
>> > > >> Suresh
>> > > >>
>> > > >>
>> > > >>
>> > > >> On Fri, Sep 11, 2015 at 9:27 AM, Radu Marias
>> > > >> <radumarias@gmail.com>
>> > > >> wrote:
>> > > >>
>> > > >> > Hi,
>> > > >> >
>> > > >> > I'm trying to run HammerDB TPC-C from this tutorial
>> > > >> > http://hammerora.sourceforge.net/hammerdb_quickstart_trafodion.
>> > > >> > pd
>> > > >> > f
>> > > >> >
>> > > >> > I have this environment:
>> > > >> >
>> > > >> >
>> > > >> > *CentOS release 6.7 (Final)*
>> > > >> > *Ambari 2.1.1*
>> > > >> > *HDP 2.2*
>> > > >> > *trafodion-20150828_0830*
>> > > >> >
>> > > >> > *HammerDB-2.18*
>> > > >> >
>> > > >> > *java version "1.7.0_79"*
>> > > >> >
>> > > >> > On *Schema Build* I get errors when stored procedures are
>> created.
>> > > >> >
>> > > >> > *This is from the hammerdb logs:*
>> > > >> >
>> > > >> > Vuser 1:CREATING TPCC STORED PROCEDURES Vuser 1:Failed to
>> > > >> > create library
>> > > >> /home/trafodion/HammerDB-2.18/NEWORDER.jar
>> > > >> >
>> > > >> > *This is what I see in the UI (attached is also a screenshot):*
>> > > >> >
>> > > >> > loading history file ... 0 events added Main console display
>> > > >> > active
>> > > >> > (Tcl8.6.0 / Tk8.6.0) The xml in config.xml is well-formed,
>> > > >> > applying variables Error in Virtual User 1: [Trafodion ODBC
>> > > >> > Driver][Trafodion Database] SQL
>> > > >> > ERROR:*** ERROR[1382] JAR or DLL file
>> > > >> > /home/trafodion/HammerDB-2.18/NEWORDER.jar not found.
>> > > >> > [2015-09-11 07:50:53]À€ç¿™À€éŽˆç»„ç¿™À€éŽ„组翙À€éŽ€ç»„ç¿™À€é
>> > > >> > 组翙À€é 组翙À€éŒ¸ç»„ç¿™À€å
¦è šç¿™À€ìªˆÊŒÀ€À€ì¹°Ê‰À€À€é
>> > > >> >
>> > > >> 组翙À€À€À€À€À€À€À€À€À€À€À€À€À€À€À€À€À€À€À€À€À€À€À€À€À€À€À€À€À€Õ
>> > > >> €Ë
>> > > >> ®À
>> > > >> €À€ìŸ
>> > > >> > Ì…À€À€À€À€À€À€À€À€À€À€ï¿¿ï¿¿À€À€ì«
ï¿¿
>> > > >> À€À€À€À€À€À€À€À€À€À€À€À€À€À€À€À€À€é
>> > > >> > °À€À€À€ì² ʉÀ€À€îˆ°Ò»À€À€éŸ¼ç«
ç¿™À€ë¾°Ç´À€À€å–€Ò À€À€ç½
>> > > >> > Ë—À€À€ì² ʉÀ€À€é’€ç»„ç¿™À€ã
µç« ç¿™À€é‘ 组翙À€ìµ ʉÀ€À€é °ç»„
>> > > >> > (executing the statement)
>> > > >> > (HammerDB-2.18) 1 %
>> > > >> >
>> > > >> > By looking in the tpcc schema I see some tables are created
and
>> > > >> > are populated with data:
>> > > >> >
>> > > >> > >>set schema tpcc;
>> > > >> >
>> > > >> > --- SQL operation complete.
>> > > >> > >>get tables;
>> > > >> >
>> > > >> > Tables in Schema TRAFODION.TPCC ===============================
>> > > >> >
>> > > >> > CUSTOMER
>> > > >> > DISTRICT
>> > > >> > HISTORY
>> > > >> > ITEM
>> > > >> > NEW_ORDER
>> > > >> > ORDERS
>> > > >> > ORDER_LINE
>> > > >> > STOCK
>> > > >> > WAREHOUSE
>> > > >> >
>> > > >> > The file '*/home/trafodion/HammerDB-2.18/NEWORDER.jar*' exists,
>> > > >> > attached is the NEWORDER.java
>> > > >> >
>> > > >> > *When trying to create the stored procedure from sqlci I
get
>> > > >> > this:*
>> > > >> >
>> > > >> > set schema tpcc;
>> > > >> > create library testrs file
>> > > >> > '/home/trafodion/HammerDB-2.18/NEWORDER.jar';
>> > > >> > create procedure NEWORD()
>> > > >> >        language java
>> > > >> >        parameter style java
>> > > >> >        external name 'NEWORDER.NEWORD'
>> > > >> >        dynamic result sets 1
>> > > >> >        library testrs;
>> > > >> >
>> > > >> > *** ERROR[11239] No compatible Java methods named 'NEWORD'
were
>> > > >> > found in Java class 'NEWORDER'.
>> > > >> >
>> > > >> > *** ERROR[1231] User-defined routine TRAFODION.TPCC.NEWORDER
>> > > >> > could not
>> > > >> be
>> > > >> > created.
>> > > >> >
>> > > >> > Is there an API change on methods for stored procedures in
the
>> > > >> > latest trafodion and hammerdb uses the older syntax? This
is
>> > > >> > the
>> > > >> > method:
>> > > >> >
>> > > >> > public static void NEWORD (int no_w_id, int no_max_w_id,
int
>> > > >> > no_d_id,
>> > > >> int
>> > > >> > no_c_id, int no_o_ol_cnt, BigDecimal[] no_c_discount, String[]
>> > > >> no_c_last,
>> > > >> > String[] no_c_credit, BigDecimal[] no_d_tax, BigDecimal[]
>> > > >> > no_w_tax,
>> > > >> int[]
>> > > >> > no_d_next_o_id, Timestamp tstamp, ResultSet[] opres) throws
>> > > >> > SQLException
>> > > >> >
>> > > >> > Also If I disable "*Build Java Stored Procedures Locally*"
from
>> > > >> > hammerdb I  get:
>> > > >> >
>> > > >> > Vuser 1:CREATING TPCC INDEXES
>> > > >> > Vuser 1:Creating Index CUSTOMER_I2...
>> > > >> > Vuser 1:Failed to Create Index
>> > > >> >
>> > > >> > --
>> > > >> > And in the end, it's not the years in your life that count.
>> > > >> > It's the
>> > > >> life
>> > > >> > in your years.
>> > > >> >
>> > > >>
>> > > >
>> > > >
>> > > >
>> > > > --
>> > > > And in the end, it's not the years in your life that count. It's
>> > > > the life in your years.
>> > > >
>> > >
>> > >
>> > >
>> > > --
>> > > And in the end, it's not the years in your life that count. It's the
>> > > life in your years.
>> > >
>> >
>> >
>> >
>> > --
>> > And in the end, it's not the years in your life that count. It's the
>> > life in your years.
>> >
>>
>>
>>
>> --
>> And in the end, it's not the years in your life that count. It's the life
>> in
>> your years.
>>
>
>
>
> --
> And in the end, it's not the years in your life that count. It's the life
> in your years.
>



-- 
And in the end, it's not the years in your life that count. It's the life
in your years.

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message