trafodion-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Dave Birdsall <dave.birds...@esgyn.com>
Subject RE: problem with update statistics
Date Thu, 24 Mar 2016 22:00:26 GMT
Hi,



Moving this over to the dev list (but still copying user).



Another thing that might be interesting is to see what UPDATE STATISTICS
was doing at the time of the failure.



To do that, before you run UPDATE STATISTICS, turn logging on:



Cqd ustat_log ‘<your file name here>’ ;



Update statistics log on;



<your update statistics command here>



If it fails again, check the log in <your file name here> to see what
UPDATE STATISTICS was doing. You may be able to reproduce the problem
outside of UPDATE STATISTICS by just running the statement seen there.



Dave



*From:* Dave Birdsall [mailto:dave.birdsall@esgyn.com]
*Sent:* Thursday, March 24, 2016 2:58 PM
*To:* 'user@trafodion.incubator.apache.org' <
user@trafodion.incubator.apache.org>
*Subject:* RE: problem with update statistics



Hi,



I’ve seen this when the HMaster has died on a workstation using the
standalone Hadoop. Perhaps you’d see the same behavior if a RegionServer
died on a cluster, I don’t know.



You may want to check your HBase logs to see if there are any clues there.



Given that you’re developing a Trafodion feature, it is possible that
something Trafodion did or said to HBase made it blush. J



Dave



*From:* Eric Owhadi [mailto:eric.owhadi@esgyn.com <eric.owhadi@esgyn.com>]
*Sent:* Thursday, March 24, 2016 2:32 PM
*To:* user@trafodion.incubator.apache.org
*Subject:* problem with update statistics



Hi Trafodioneers,

I am trying to create a 10M row table, 1KB each row 10 partitions to test a
new feature (parallelScan).

But on my dev (so single node), I stumble on update statistics error:



update statistics for table t132 on every column sample;

*** ERROR[9200] UPDATE STATISTICS for table TRAFODION.SCH.T132 encountered
an error (8448) from statement HSCursor::fetchRowset().



*** ERROR[8448] Unable to access Hbase interface. Call to
ExpHbaseInterface::nextRow returned error HBASE_ACCESS_ERROR(-706). Cause:

java.util.concurrent.ExecutionException:
org.apache.hadoop.hbase.client.RetriesExhaustedException: Failed after
attempts=35, exceptions:

Thu Mar 24 21:01:52 GMT 2016,
org.apache.hadoop.hbase.client.RpcRetryingCaller@7c24d3f1,
java.io.IOException: Call to edevxx.xxxxx.local/10.1.xx.xx:38868 failed on
local exception: java.io.EOFException

Thu Mar 24 21:02:11 GMT 2016,
org.apache.hadoop.hbase.client.RpcRetryingCaller@7c24d3f1,
java.io.IOException: Enable/Disable failed

Thu Mar 24 21:02:30 GMT 2016,
org.apache.hadoop.hbase.client.RpcRetryingCaller@7c24d3f1,
java.io.IOException: Enable/Disable failed

Thu Mar 24 21:02:51 GMT 2016,
org.apache.hadoop.hbase.client.RpcRetryingCaller@7c24d3f1,
java.io.IOException: Enable/Disable failed

Thu Mar 24 21:03:12 GMT 2016,
org.apache.hadoop.hbase.client.RpcRetryingCaller@7c24d3f1,
java.io.IOException: Enable/Disable failed

Thu Mar 24 21:03:33 GMT 2016,
org.apache.hadoop.hbase.client.RpcRetryingCaller@7c24d3f1,
java.io.IOException: Enable/Disable failed

Thu Mar 24 21:04:03 GMT 2016,
org.apache.hadoop.hbase.client.RpcRetryingCaller@7c24d3f1,
java.io.IOException: Enable/Disable failed

Thu Mar 24 21:04:32 GMT 2016,
org.apache.hadoop.hbase.client.RpcRetryingCaller@7c24d3f1,
java.io.IOException: Enable/Disable failed

Thu Mar 24 21:05:00 GMT 2016,
org.apache.hadoop.hbase.client.RpcRetryingCaller@7c24d3f1,
java.io.IOException: Enable/Disable failed

Thu Mar 24 21:05:27 GMT 2016,
org.apache.hadoop.hbase.client.RpcRetryingCaller@7c24d3f1,
java.io.IOException: Enable/Disable failed

Thu Mar 24 21:06:04 GMT 2016,
org.apache.hadoop.hbase.client.RpcRetryingCaller@7c24d3f1,
java.io.IOException: Enable/Disable failed

Thu Mar 24 21:06:42 GMT 2016,
org.apache.hadoop.hbase.client.RpcRetryingCaller@7c24d3f1,
java.io.IOException: Enable/Disable failed

Thu Mar 24 21:07:20 GMT 2016,
org.apache.hadoop.hbase.client.RpcRetryingCaller@7c24d3f1,
java.io.IOException: Enable/Disable failed

Thu Mar 24 21:08:00 GMT 2016,
org.apache.hadoop.hbase.client.RpcRetryingCaller@7c24d3f1,
java.io.IOException: Enable/Disable failed

Thu Mar 24 21:08:38 GMT 2016,
org.apache.hadoop.hbase.client.RpcRetryingCaller@7c24d3f1,
java.io.IOException: Enable/Disable failed

Thu Mar 24 21:09:18 GMT 2016,
org.apache.hadoop.hbase.client.RpcRetryingCaller@7c24d3f1,
java.io.IOException: Enable/Disable failed

Thu Mar 24 21:09:56 GMT 2016,
org.apache.hadoop.hbase.client.RpcRetryingCaller@7c24d3f1,
java.io.IOException: Enable/Disable failed

Thu Mar 24 21:10:34 GMT 2016,
org.apache.hadoop.hbase.client.RpcRetryingCaller@7c24d3f1,
java.io.IOException: Enable/Disable failed

Thu Mar 24 21:11:13 GMT 2016,
org.apache.hadoop.hbase.client.RpcRetryingCaller@7c24d3f1,
java.io.IOException: Enable/Disable failed

Thu Mar 24 21:11:52 GMT 2016,
org.apache.hadoop.hbase.client.RpcRetryingCaller@7c24d3f1,
java.io.IOException: Enable/Disable failed

Thu Mar 24 21:12:33 GMT 2016,
org.apache.hadoop.hbase.client.RpcRetryingCaller@7c24d3f1,
java.io.IOException: Enable/Disable failed

Thu Mar 24 21:13:11 GMT 2016,
org.apache.hadoop.hbase.client.RpcRetryingCaller@7c24d3f1,
java.io.IOException: Enable/Disable failed

Thu Mar 24 21:13:51 GMT 2016,
org.apache.hadoop.hbase.client.RpcRetryingCaller@7c24d3f1,
java.io.IOException: Enable/Disable failed

Thu Mar 24 21:14:31 GMT 2016,
org.apache.hadoop.hbase.client.RpcRetryingCaller@7c24d3f1,
java.io.IOException: Enable/Disable failed

Thu Mar 24 21:15:08 GMT 2016,
org.apache.hadoop.hbase.client.RpcRetryingCaller@7c24d3f1,
java.io.IOException: Enable/Disable failed

Thu Mar 24 21:15:47 GMT 2016,
org.apache.hadoop.hbase.client.RpcRetryingCaller@7c24d3f1,
java.io.IOException: Enable/Disable failed

Thu Mar 24 21:16:29 GMT 2016,
org.apache.hadoop.hbase.client.RpcRetryingCaller@7c24d3f1,
java.io.IOException: Enable/Disable failed

Thu Mar 24 21:17:07 GMT 2016,
org.apache.hadoop.hbase.client.RpcRetryingCaller@7c24d3f1,
java.io.IOException: Enable/Disable failed

Thu Mar 24 21:17:45 GMT 2016, org.apache.hadoop.hbase.clien'��k




*** ERROR[9200] UPDATE STATISTICS for table TRAFODION.SCH.T132 encountered
an error (8448) from statement READ_COLS_INTO_MEM.



*** ERROR[8448] Unable to access Hbase interface. Call to
ExpHbaseInterface::nextRow returned error HBASE_ACCESS_ERROR(-706). Cause:

java.util.concurrent.ExecutionException:
org.apache.hadoop.hbase.client.RetriesExhaustedException: Failed after
attempts=35, exceptions:

Thu Mar 24 21:01:52 GMT 2016,
org.apache.hadoop.hbase.client.RpcRetryingCaller@7c24d3f1,
java.io.IOException: Call to xxxxx.xxxxx.local/10.1.xx.xx:38868 failed on
local exception: java.io.EOFException

Thu Mar 24 21:02:11 GMT 2016,
org.apache.hadoop.hbase.client.RpcRetryingCaller@7c24d3f1,
java.io.IOException: Enable/Disable failed

Thu Mar 24 21:02:30 GMT 2016,
org.apache.hadoop.hbase.client.RpcRetryingCaller@7c24d3f1,
java.io.IOException: Enable/Disable failed

Thu Mar 24 21:02:51 GMT 2016,
org.apache.hadoop.hbase.client.RpcRetryingCaller@7c24d3f1,
java.io.IOException: Enable/Disable failed

Thu Mar 24 21:03:12 GMT 2016,
org.apache.hadoop.hbase.client.RpcRetryingCaller@7c24d3f1,
java.io.IOException: Enable/Disable failed

Thu Mar 24 21:03:33 GMT 2016,
org.apache.hadoop.hbase.client.RpcRetryingCaller@7c24d3f1,
java.io.IOException: Enable/Disable failed

Thu Mar 24 21:04:03 GMT 2016,
org.apache.hadoop.hbase.client.RpcRetryingCaller@7c24d3f1,
java.io.IOException: Enable/Disable failed

Thu Mar 24 21:04:32 GMT 2016,
org.apache.hadoop.hbase.client.RpcRetryingCaller@7c24d3f1,
java.io.IOException: Enable/Disable failed

Thu Mar 24 21:05:00 GMT 2016,
org.apache.hadoop.hbase.client.RpcRetryingCaller@7c24d3f1,
java.io.IOException: Enable/Disable failed

Thu Mar 24 21:05:27 GMT 2016,
org.apache.hadoop.hbase.client.RpcRetryingCaller@7c24d3f1,
java.io.IOException: Enable/Disable failed

Thu Mar 24 21:06:04 GMT 2016,
org.apache.hadoop.hbase.client.RpcRetryingCaller@7c24d3f1,
java.io.IOException: Enable/Disable failed

Thu Mar 24 21:06:42 GMT 2016,
org.apache.hadoop.hbase.client.RpcRetryingCaller@7c24d3f1,
java.io.IOException: Enable/Disable failed

Thu Mar 24 21:07:20 GMT 2016,
org.apache.hadoop.hbase.client.RpcRetryingCaller@7c24d3f1,
java.io.IOException: Enable/Disable failed

Thu Mar 24 21:08:00 GMT 2016,
org.apache.hadoop.hbase.client.RpcRetryingCaller@7c24d3f1,
java.io.IOException: Enable/Disable failed

Thu Mar 24 21:08:38 GMT 2016,
org.apache.hadoop.hbase.client.RpcRetryingCaller@7c24d3f1,
java.io.IOException: Enable/Disable failed

Thu Mar 24 21:09:18 GMT 2016,
org.apache.hadoop.hbase.client.RpcRetryingCaller@7c24d3f1,
java.io.IOException: Enable/Disable failed

Thu Mar 24 21:09:56 GMT 2016,
org.apache.hadoop.hbase.client.RpcRetryingCaller@7c24d3f1,
java.io.IOException: Enable/Disable failed

Thu Mar 24 21:10:34 GMT 2016,
org.apache.hadoop.hbase.client.RpcRetryingCaller@7c24d3f1,
java.io.IOException: Enable/Disable failed

Thu Mar 24 21:11:13 GMT 2016,
org.apache.hadoop.hbase.client.RpcRetryingCaller@7c24d3f1,
java.io.IOException: Enable/Disable failed

Thu Mar 24 21:11:52 GMT 2016,
org.apache.hadoop.hbase.client.RpcRetryingCaller@7c24d3f1,
java.io.IOException: Enable/Disable failed

Thu Mar 24 21:12:33 GMT 2016,
org.apache.hadoop.hbase.client.RpcRetryingCaller@7c24d3f1,
java.io.IOException: Enable/Disable failed

Thu Mar 24 21:13:11 GMT 2016,
org.apache.hadoop.hbase.client.RpcRetryingCaller@7c24d3f1,
java.io.IOException: Enable/Disable failed

Thu Mar 24 21:13:51 GMT 2016,
org.apache.hadoop.hbase.client.RpcRetryingCaller@7c24d3f1,
java.io.IOException: Enable/Disable failed

Thu Mar 24 21:14:31 GMT 2016,
org.apache.hadoop.hbase.client.RpcRetryingCaller@7c24d3f1,
java.io.IOException: Enable/Disable failed

Thu Mar 24 21:15:08 GMT 2016,
org.apache.hadoop.hbase.client.RpcRetryingCaller@7c24d3f1,
java.io.IOException: Enable/Disable failed

Thu Mar 24 21:15:47 GMT 2016,
org.apache.hadoop.hbase.client.RpcRetryingCaller@7c24d3f1,
java.io.IOException: Enable/Disable failed

Thu Mar 24 21:16:29 GMT 2016,
org.apache.hadoop.hbase.client.RpcRetryingCaller@7c24d3f1,
java.io.IOException: Enable/Disable failed

Thu Mar 24 21:17:07 GMT 2016,
org.apache.hadoop.hbase.client.RpcRetryingCaller@7c24d3f1,
java.io.IOException: Enable/Disable failed

Thu Mar 24 21:17:45 GMT 2016, org.apache.hadoop.hbase.clien'��k





The table:

create table t132helper (a int not null, primary key(a));

insert into t132helper values(1);

create table t132 (k1 int not null, k2 int not null, a int not null, b int
not null,

     c
char(1000),
primary key (k1,k2))

     SALT USING 10 PARTITIONS

       ON (k1, k2)

      ATTRIBUTES ALIGNED FORMAT ;

upsert using load

into t132

  select  x1000*1000+ x10000*10000 + x100000*100000 + x1000000*1000000,

          x1+x10*10+x100*100,

          x1+x10*10+x100*100+ x1000*1000+ x10000*10000 + x100000*100000 +
x1000000*1000000,

          x1+x10*10+x100*100+ x1000*1000+ x10000*10000 + x100000*100000 +
x1000000*1000000,

          'yo bro'



  from t132helper

transpose 0,1,2,3,4,5,6,7,8,9 as x1

transpose 0,1,2,3,4,5,6,7,8,9 as x10

transpose 0,1,2,3,4,5,6,7,8,9 as x100

transpose 0,1,2,3,4,5,6,7,8,9 as x1000

transpose 0,1,2,3,4,5,6,7,8,9 as x10000

transpose 0,1,2,3,4,5,6,7,8,9 as x100000

transpose 0,1,2,3,4,5,6,7,8,9 as x1000000;

update statistics for table t132 on every column sample;





is that a known problem? A workaround available?

Eric

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message