hbase-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Ted Yu <yuzhih...@gmail.com>
Subject Re: 答复: how to Dynamic load of Coprocessors
Date Fri, 22 Jul 2016 16:36:53 GMT
The pasted log was from master, right ?

Can you take a look at region server involved ?

Cheers

On Fri, Jul 22, 2016 at 9:23 AM, Ma, Sheng-Chen (Aven) <
shengchen.ma@esgyn.cn> wrote:

> Hi Ted:
> Following is part of log. I have changed log level to TRACE. But still
> can’t find any clue.
>
> 2016-07-22 12:03:38,261 DEBUG [ProcedureExecutor-1]
> procedure2.ProcedureExecutor: Procedure completed in 1.6310sec:
> DisableTableProcedure (table=testTbl) id=109 owner=shengchen.ma state=F
> INISHED
> 2016-07-22 12:03:38,866 TRACE
> [RpcServer.reader=7,bindAddress=aven01.novalocal,port=39479] ipc.RpcServer:
> RequestHeader call_id: 14 method_name: "IsMasterRunning" request_param:
> true totalRequestSize: 23 bytes
> 2016-07-22 12:03:38,866 TRACE
> [B.defaultRpcServer.handler=24,queue=0,port=39479] ipc.RpcServer: callId:
> 14 service: MasterService methodName: IsMasterRunning size: 23 connection:
> 192.168.3.10:49752 executing as shengchen.ma
> 2016-07-22 12:03:38,866 TRACE
> [B.defaultRpcServer.handler=24,queue=0,port=39479] ipc.RpcServer: callId:
> 14 service: MasterService methodName: IsMasterRunning size: 23 connection:
> 192.168.3.10:49752 param: TODO: class
> org.apache.hadoop.hbase.protobuf.generated.MasterProtos$IsMasterRunningRequest
> connection: 192.168.3.10:49752, response is_master_running: true
> queueTime: 0 processingTime: 0 totalTime: 0
> 2016-07-22 12:03:38,878 TRACE
> [RpcServer.reader=7,bindAddress=aven01.novalocal,port=39479] ipc.RpcServer:
> RequestHeader call_id: 15 method_name: "getProcedureResult" request_param:
> true totalRequestSize: 28 bytes
> 2016-07-22 12:03:38,878 TRACE
> [B.defaultRpcServer.handler=7,queue=1,port=39479] ipc.RpcServer: callId: 15
> service: MasterService methodName: getProcedureResult size: 28 connection:
> 192.168.3.10:49752 executing as shengchen.ma
> 2016-07-22 12:03:38,878 DEBUG
> [B.defaultRpcServer.handler=7,queue=1,port=39479] master.MasterRpcServices:
> Checking to see if procedure is done procId=109
> 2016-07-22 12:03:38,878 TRACE
> [B.defaultRpcServer.handler=7,queue=1,port=39479] ipc.RpcServer: callId: 15
> service: MasterService methodName: getProcedureResult size: 28 connection:
> 192.168.3.10:49752 param: TODO: class
> org.apache.hadoop.hbase.protobuf.generated.MasterProtos$GetProcedureResultRequest
> connection: 192.168.3.10:49752, response state: FINISHED start_time:
> 1469189016526 last_update: 1469189018157 queueTime: 0 processingTime: 0
> totalTime: 0
> 2016-07-22 12:03:38,888 TRACE
> [RpcServer.reader=7,bindAddress=aven01.novalocal,port=39479] ipc.RpcServer:
> RequestHeader call_id: 16 method_name: "IsMasterRunning" request_param:
> true totalRequestSize: 23 bytes
> 2016-07-22 12:03:38,888 TRACE
> [B.defaultRpcServer.handler=9,queue=0,port=39479] ipc.RpcServer: callId: 16
> service: MasterService methodName: IsMasterRunning size: 23 connection:
> 192.168.3.10:49752 executing as shengchen.ma
> 2016-07-22 12:03:38,888 TRACE
> [B.defaultRpcServer.handler=9,queue=0,port=39479] ipc.RpcServer: callId: 16
> service: MasterService methodName: IsMasterRunning size: 23 connection:
> 192.168.3.10:49752 param: TODO: class
> org.apache.hadoop.hbase.protobuf.generated.MasterProtos$IsMasterRunningRequest
> connection: 192.168.3.10:49752, response is_master_running: true
> queueTime: 0 processingTime: 0 totalTime: 0
> 2016-07-22 12:03:39,007 TRACE
> [RpcServer.reader=7,bindAddress=aven01.novalocal,port=39479] ipc.RpcServer:
> RequestHeader call_id: 17 method_name: "ModifyTable" request_param: true
> totalRequestSize: 505 bytes
> 2016-07-22 12:03:39,007 TRACE
> [B.defaultRpcServer.handler=17,queue=2,port=39479] ipc.RpcServer: callId:
> 17 service: MasterService methodName: ModifyTable size: 505 connection:
> 192.168.3.10:49752 executing as shengchen.ma
> 2016-07-22 12:03:39,047 DEBUG
> [B.defaultRpcServer.handler=17,queue=2,port=39479]
> util.CoprocessorClassLoader: Skipping exempt class
> org.apache.hadoop.hbase.coprocessor.transactional.TestRegionEndpoint -
> delegating directly to parent
> 2016-07-22 12:03:39,048 DEBUG
> [B.defaultRpcServer.handler=17,queue=2,port=39479] ipc.RpcServer:
> B.defaultRpcServer.handler=17,queue=2,port=39479: callId: 17 service:
> MasterService methodName: ModifyTable size: 505 connection:
> 192.168.3.10:49752
> org.apache.hadoop.hbase.DoNotRetryIOException: Class
> org.apache.hadoop.hbase.coprocessor.transactional.TestRegionEndpoint cannot
> be loaded Set hbase.table.sanity.checks to false at conf or table
> descriptor if you want to bypass sanity checks
>         at
> org.apache.hadoop.hbase.master.HMaster.warnOrThrowExceptionForFailure(HMaster.java:1680)
>         at
> org.apache.hadoop.hbase.master.HMaster.sanityCheckTableDescriptor(HMaster.java:1541)
>         at
> org.apache.hadoop.hbase.master.HMaster.modifyTable(HMaster.java:2028)
>         at
> org.apache.hadoop.hbase.master.MasterRpcServices.modifyTable(MasterRpcServices.java:1170)
>         at
> org.apache.hadoop.hbase.protobuf.generated.MasterProtos$MasterService$2.callBlockingMethod(MasterProtos.java:55680)
>         at org.apache.hadoop.hbase.ipc.RpcServer.call(RpcServer.java:2170)
>         at org.apache.hadoop.hbase.ipc.CallRunner.run(CallRunner.java:109)
>         at
> org.apache.hadoop.hbase.ipc.RpcExecutor.consumerLoop(RpcExecutor.java:133)
>         at
> org.apache.hadoop.hbase.ipc.RpcExecutor$1.run(RpcExecutor.java:108)
>         at java.lang.Thread.run(Thread.java:745)
> 2016-07-22 12:03:39,622 DEBUG
> [RpcServer.reader=7,bindAddress=aven01.novalocal,port=39479] ipc.RpcServer:
> RpcServer.listener,port=39479: Caught exception while reading:Connection
> reset by peer
> 2016-07-22 12:03:39,622 DEBUG
> [RpcServer.reader=7,bindAddress=aven01.novalocal,port=39479] ipc.RpcServer:
> RpcServer.listener,port=39479: DISCONNECTING client 192.168.3.10:49752
> because read count=-1. Number of active connections: 3
> 2016-07-22 12:03:39,623 WARN  [NIOServerCxn.Factory:0.0.0.0/0.0.0.0:17570]
> server.NIOServerCnxn: Exception causing close of session 0x1561182c97e0009
> due to java.io.IOException: Connection reset by peer
> 2016-07-22 12:03:39,623 INFO  [NIOServerCxn.Factory:0.0.0.0/0.0.0.0:17570]
> server.NIOServerCnxn: Closed socket connection for client /
> 192.168.3.10:49751 which had sessionid 0x1561182c97e0009
> 2016-07-22 12:03:40,127 TRACE
> [RpcServer.reader=1,bindAddress=aven01.novalocal,port=39479] ipc.RpcServer:
> RequestHeader call_id: 5466 method_name: "RegionServerReport"
> request_param: true totalRequestSize: 5748 bytes
> 2016-07-22 12:03:40,128 TRACE
> [B.defaultRpcServer.handler=12,queue=0,port=39479] ipc.RpcServer: callId:
> 5466 service: RegionServerStatusService methodName: RegionServerReport
> size: 5.6 K connection: 192.168.0.17:49483 executing as traf
> 2016-07-22 12:03:40,128 TRACE
> [B.defaultRpcServer.handler=12,queue=0,port=39479] master.ServerManager:
> 7e8aa2d93aa716ad2068808d938f0786, existingValue=-1, completeSequenceId=-1
> 2016-07-22 12:03:40,128 TRACE
> [B.defaultRpcServer.handler=12,queue=0,port=39479] master.ServerManager:
> 7e8aa2d93aa716ad2068808d938f0786, family=mt_, existingValue=-1,
> completeSequenceId=-1
> 2016-07-22 12:03:40,128 TRACE
> [B.defaultRpcServer.handler=12,queue=0,port=39479] master.ServerManager:
> 7e8aa2d93aa716ad2068808d938f0786, family=tddlcf, existingValue=-1,
> completeSequenceId=-1
> 2016-07-22 12:03:40,128 TRACE
> [B.defaultRpcServer.handler=12,queue=0,port=39479] master.ServerManager:
> 3052710e85ab46a686a22e23bff937e2, existingValue=-1, completeSequenceId=-1
> 2016-07-22 12:03:40,128 TRACE
> [B.defaultRpcServer.handler=12,queue=0,port=39479] master.ServerManager:
> 3052710e85ab46a686a22e23bff937e2, family=cpf, existingValue=-1,
> completeSequenceId=-1
>
> -----邮件原件-----
> 发件人: Ted Yu [mailto:yuzhihong@gmail.com]
> 发送时间: 2016年7月22日 19:48
> 收件人: user@hbase.apache.org
> 主题: Re: how to Dynamic load of Coprocessors
>
> w.r.t. the DoNotRetryIOException, can you take a look at region server log
> where testTbl region(s) was hosted ?
>
> See if there is some clue why the sanity check failed.
>
> Thanks
>
> On Fri, Jul 22, 2016 at 1:12 AM, Ma, Sheng-Chen (Aven) <
> shengchen.ma@esgyn.cn> wrote:
>
> > Hi all:
> > I want to dynamic add coprocessor in order to not restart hbase.
> >
> > Following is my code:
> >         Path path = new Path("/coprocessor_jars");
> >         FileSystem fs = FileSystem.get(conf);
> >         FileStatus[] status = fs.listStatus(path);
> >         Path[] listedPaths = FileUtil.stat2Paths(status);
> >         for (Path p : listedPaths) {
> >             if(p.getName().contains("test.jar")){
> >                 hdfsPath = p;
> >             }
> >         }
> >         HBaseAdmin hadmin = new HBaseAdmin(conf);
> >         HTableDescriptor tableDesc =
> > hadmin.getTableDescriptor("testTbl".getBytes());
> >
> > tableDesc.addCoprocessor("org.apache.hadoop.hbase.coprocessor.transact
> > ional.TestRegionEndpoint",
> > hdfsPath,
> >                 Coprocessor.PRIORITY_USER, null);
> >         //
> >
> tableDesc.removeCoprocessor("org.apache.hadoop.hbase.coprocessor.transactional.TrxRegionEndpoint");
> >         for (Entry<ImmutableBytesWritable, ImmutableBytesWritable> entry
> :
> > tableDesc.getValues().entrySet()) {
> >             System.out.println(Bytes.toString(entry.getKey().get()) +
> > " = " + Bytes.toString(entry.getValue().get()));
> >         }
> >         hadmin.disableTable("testTbl".getBytes());
> >         hadmin.modifyTable("testTbl", tableDesc);
> >         hadmin.enableTable("testTbl");
> >
> > the syso print : coprocessor$1 = hdfs://
> > 192.168.0.17:17400/coprocessor_jars/test.jar|org.apache.hadoop.hbase.c
> > oprocessor.transactional.TestRegionEndpoint|1073741823|
> >
> > and the remote side return Exception:
> > org.apache.hadoop.hbase.DoNotRetryIOException:
> > org.apache.hadoop.hbase.DoNotRetryIOException: Class
> > org.apache.hadoop.hbase.coprocessor.transactional.TestRegionEndpoint
> > cannot be loaded Set hbase.table.sanity.checks to false at conf or
> > table descriptor if you want to bypass sanity checks
> >
> > I use hbase 1.2 and the test.jar is not under hbase/lib, I just put
> > the test.jar in hdfs.
> >
> > If I add test.jar to hbase/lib but not restart hbase, the upon code
> > still throw same exception.
> > If I add test.jar to hbase/lib and restart hbase, the upon code will
> > exec successful.
> >
> > But my requirement is not restart hbase.
> >
> > Is there someone can give me a favor.
> >
> > Thanks
> >
> >
> >
>

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message