hbase-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Ma, Sheng-Chen (Aven)" <shengchen...@esgyn.cn>
Subject 答复: how to Dynamic load of Coprocessors
Date Fri, 22 Jul 2016 16:23:06 GMT
Hi Ted:
Following is part of log. I have changed log level to TRACE. But still can’t find any clue.

2016-07-22 12:03:38,261 DEBUG [ProcedureExecutor-1] procedure2.ProcedureExecutor: Procedure
completed in 1.6310sec: DisableTableProcedure (table=testTbl) id=109 owner=shengchen.ma state=F
INISHED
2016-07-22 12:03:38,866 TRACE [RpcServer.reader=7,bindAddress=aven01.novalocal,port=39479]
ipc.RpcServer: RequestHeader call_id: 14 method_name: "IsMasterRunning" request_param: true
totalRequestSize: 23 bytes
2016-07-22 12:03:38,866 TRACE [B.defaultRpcServer.handler=24,queue=0,port=39479] ipc.RpcServer:
callId: 14 service: MasterService methodName: IsMasterRunning size: 23 connection: 192.168.3.10:49752
executing as shengchen.ma
2016-07-22 12:03:38,866 TRACE [B.defaultRpcServer.handler=24,queue=0,port=39479] ipc.RpcServer:
callId: 14 service: MasterService methodName: IsMasterRunning size: 23 connection: 192.168.3.10:49752
param: TODO: class org.apache.hadoop.hbase.protobuf.generated.MasterProtos$IsMasterRunningRequest
connection: 192.168.3.10:49752, response is_master_running: true queueTime: 0 processingTime:
0 totalTime: 0
2016-07-22 12:03:38,878 TRACE [RpcServer.reader=7,bindAddress=aven01.novalocal,port=39479]
ipc.RpcServer: RequestHeader call_id: 15 method_name: "getProcedureResult" request_param:
true totalRequestSize: 28 bytes
2016-07-22 12:03:38,878 TRACE [B.defaultRpcServer.handler=7,queue=1,port=39479] ipc.RpcServer:
callId: 15 service: MasterService methodName: getProcedureResult size: 28 connection: 192.168.3.10:49752
executing as shengchen.ma
2016-07-22 12:03:38,878 DEBUG [B.defaultRpcServer.handler=7,queue=1,port=39479] master.MasterRpcServices:
Checking to see if procedure is done procId=109
2016-07-22 12:03:38,878 TRACE [B.defaultRpcServer.handler=7,queue=1,port=39479] ipc.RpcServer:
callId: 15 service: MasterService methodName: getProcedureResult size: 28 connection: 192.168.3.10:49752
param: TODO: class org.apache.hadoop.hbase.protobuf.generated.MasterProtos$GetProcedureResultRequest
connection: 192.168.3.10:49752, response state: FINISHED start_time: 1469189016526 last_update:
1469189018157 queueTime: 0 processingTime: 0 totalTime: 0
2016-07-22 12:03:38,888 TRACE [RpcServer.reader=7,bindAddress=aven01.novalocal,port=39479]
ipc.RpcServer: RequestHeader call_id: 16 method_name: "IsMasterRunning" request_param: true
totalRequestSize: 23 bytes
2016-07-22 12:03:38,888 TRACE [B.defaultRpcServer.handler=9,queue=0,port=39479] ipc.RpcServer:
callId: 16 service: MasterService methodName: IsMasterRunning size: 23 connection: 192.168.3.10:49752
executing as shengchen.ma
2016-07-22 12:03:38,888 TRACE [B.defaultRpcServer.handler=9,queue=0,port=39479] ipc.RpcServer:
callId: 16 service: MasterService methodName: IsMasterRunning size: 23 connection: 192.168.3.10:49752
param: TODO: class org.apache.hadoop.hbase.protobuf.generated.MasterProtos$IsMasterRunningRequest
connection: 192.168.3.10:49752, response is_master_running: true queueTime: 0 processingTime:
0 totalTime: 0
2016-07-22 12:03:39,007 TRACE [RpcServer.reader=7,bindAddress=aven01.novalocal,port=39479]
ipc.RpcServer: RequestHeader call_id: 17 method_name: "ModifyTable" request_param: true totalRequestSize:
505 bytes
2016-07-22 12:03:39,007 TRACE [B.defaultRpcServer.handler=17,queue=2,port=39479] ipc.RpcServer:
callId: 17 service: MasterService methodName: ModifyTable size: 505 connection: 192.168.3.10:49752
executing as shengchen.ma
2016-07-22 12:03:39,047 DEBUG [B.defaultRpcServer.handler=17,queue=2,port=39479] util.CoprocessorClassLoader:
Skipping exempt class org.apache.hadoop.hbase.coprocessor.transactional.TestRegionEndpoint
- delegating directly to parent
2016-07-22 12:03:39,048 DEBUG [B.defaultRpcServer.handler=17,queue=2,port=39479] ipc.RpcServer:
B.defaultRpcServer.handler=17,queue=2,port=39479: callId: 17 service: MasterService methodName:
ModifyTable size: 505 connection: 192.168.3.10:49752
org.apache.hadoop.hbase.DoNotRetryIOException: Class org.apache.hadoop.hbase.coprocessor.transactional.TestRegionEndpoint
cannot be loaded Set hbase.table.sanity.checks to false at conf or table descriptor if you
want to bypass sanity checks
        at org.apache.hadoop.hbase.master.HMaster.warnOrThrowExceptionForFailure(HMaster.java:1680)
        at org.apache.hadoop.hbase.master.HMaster.sanityCheckTableDescriptor(HMaster.java:1541)
        at org.apache.hadoop.hbase.master.HMaster.modifyTable(HMaster.java:2028)
        at org.apache.hadoop.hbase.master.MasterRpcServices.modifyTable(MasterRpcServices.java:1170)
        at org.apache.hadoop.hbase.protobuf.generated.MasterProtos$MasterService$2.callBlockingMethod(MasterProtos.java:55680)
        at org.apache.hadoop.hbase.ipc.RpcServer.call(RpcServer.java:2170)
        at org.apache.hadoop.hbase.ipc.CallRunner.run(CallRunner.java:109)
        at org.apache.hadoop.hbase.ipc.RpcExecutor.consumerLoop(RpcExecutor.java:133)
        at org.apache.hadoop.hbase.ipc.RpcExecutor$1.run(RpcExecutor.java:108)
        at java.lang.Thread.run(Thread.java:745)
2016-07-22 12:03:39,622 DEBUG [RpcServer.reader=7,bindAddress=aven01.novalocal,port=39479]
ipc.RpcServer: RpcServer.listener,port=39479: Caught exception while reading:Connection reset
by peer
2016-07-22 12:03:39,622 DEBUG [RpcServer.reader=7,bindAddress=aven01.novalocal,port=39479]
ipc.RpcServer: RpcServer.listener,port=39479: DISCONNECTING client 192.168.3.10:49752 because
read count=-1. Number of active connections: 3
2016-07-22 12:03:39,623 WARN  [NIOServerCxn.Factory:0.0.0.0/0.0.0.0:17570] server.NIOServerCnxn:
Exception causing close of session 0x1561182c97e0009 due to java.io.IOException: Connection
reset by peer
2016-07-22 12:03:39,623 INFO  [NIOServerCxn.Factory:0.0.0.0/0.0.0.0:17570] server.NIOServerCnxn:
Closed socket connection for client /192.168.3.10:49751 which had sessionid 0x1561182c97e0009
2016-07-22 12:03:40,127 TRACE [RpcServer.reader=1,bindAddress=aven01.novalocal,port=39479]
ipc.RpcServer: RequestHeader call_id: 5466 method_name: "RegionServerReport" request_param:
true totalRequestSize: 5748 bytes
2016-07-22 12:03:40,128 TRACE [B.defaultRpcServer.handler=12,queue=0,port=39479] ipc.RpcServer:
callId: 5466 service: RegionServerStatusService methodName: RegionServerReport size: 5.6 K
connection: 192.168.0.17:49483 executing as traf
2016-07-22 12:03:40,128 TRACE [B.defaultRpcServer.handler=12,queue=0,port=39479] master.ServerManager:
7e8aa2d93aa716ad2068808d938f0786, existingValue=-1, completeSequenceId=-1
2016-07-22 12:03:40,128 TRACE [B.defaultRpcServer.handler=12,queue=0,port=39479] master.ServerManager:
7e8aa2d93aa716ad2068808d938f0786, family=mt_, existingValue=-1, completeSequenceId=-1
2016-07-22 12:03:40,128 TRACE [B.defaultRpcServer.handler=12,queue=0,port=39479] master.ServerManager:
7e8aa2d93aa716ad2068808d938f0786, family=tddlcf, existingValue=-1, completeSequenceId=-1
2016-07-22 12:03:40,128 TRACE [B.defaultRpcServer.handler=12,queue=0,port=39479] master.ServerManager:
3052710e85ab46a686a22e23bff937e2, existingValue=-1, completeSequenceId=-1
2016-07-22 12:03:40,128 TRACE [B.defaultRpcServer.handler=12,queue=0,port=39479] master.ServerManager:
3052710e85ab46a686a22e23bff937e2, family=cpf, existingValue=-1, completeSequenceId=-1

-----邮件原件-----
发件人: Ted Yu [mailto:yuzhihong@gmail.com] 
发送时间: 2016年7月22日 19:48
收件人: user@hbase.apache.org
主题: Re: how to Dynamic load of Coprocessors

w.r.t. the DoNotRetryIOException, can you take a look at region server log where testTbl region(s)
was hosted ?

See if there is some clue why the sanity check failed.

Thanks

On Fri, Jul 22, 2016 at 1:12 AM, Ma, Sheng-Chen (Aven) < shengchen.ma@esgyn.cn> wrote:

> Hi all:
> I want to dynamic add coprocessor in order to not restart hbase.
>
> Following is my code:
>         Path path = new Path("/coprocessor_jars");
>         FileSystem fs = FileSystem.get(conf);
>         FileStatus[] status = fs.listStatus(path);
>         Path[] listedPaths = FileUtil.stat2Paths(status);
>         for (Path p : listedPaths) {
>             if(p.getName().contains("test.jar")){
>                 hdfsPath = p;
>             }
>         }
>         HBaseAdmin hadmin = new HBaseAdmin(conf);
>         HTableDescriptor tableDesc =
> hadmin.getTableDescriptor("testTbl".getBytes());
>
> tableDesc.addCoprocessor("org.apache.hadoop.hbase.coprocessor.transact
> ional.TestRegionEndpoint",
> hdfsPath,
>                 Coprocessor.PRIORITY_USER, null);
>         //
> tableDesc.removeCoprocessor("org.apache.hadoop.hbase.coprocessor.transactional.TrxRegionEndpoint");
>         for (Entry<ImmutableBytesWritable, ImmutableBytesWritable> entry :
> tableDesc.getValues().entrySet()) {
>             System.out.println(Bytes.toString(entry.getKey().get()) + 
> " = " + Bytes.toString(entry.getValue().get()));
>         }
>         hadmin.disableTable("testTbl".getBytes());
>         hadmin.modifyTable("testTbl", tableDesc);
>         hadmin.enableTable("testTbl");
>
> the syso print : coprocessor$1 = hdfs:// 
> 192.168.0.17:17400/coprocessor_jars/test.jar|org.apache.hadoop.hbase.c
> oprocessor.transactional.TestRegionEndpoint|1073741823|
>
> and the remote side return Exception:
> org.apache.hadoop.hbase.DoNotRetryIOException:
> org.apache.hadoop.hbase.DoNotRetryIOException: Class 
> org.apache.hadoop.hbase.coprocessor.transactional.TestRegionEndpoint 
> cannot be loaded Set hbase.table.sanity.checks to false at conf or 
> table descriptor if you want to bypass sanity checks
>
> I use hbase 1.2 and the test.jar is not under hbase/lib, I just put 
> the test.jar in hdfs.
>
> If I add test.jar to hbase/lib but not restart hbase, the upon code 
> still throw same exception.
> If I add test.jar to hbase/lib and restart hbase, the upon code will 
> exec successful.
>
> But my requirement is not restart hbase.
>
> Is there someone can give me a favor.
>
> Thanks
>
>
>
Mime
View raw message