From user-return-3790-apmail-phoenix-user-archive=phoenix.apache.org@phoenix.apache.org Wed Sep 16 01:44:34 2015 Return-Path: X-Original-To: apmail-phoenix-user-archive@minotaur.apache.org Delivered-To: apmail-phoenix-user-archive@minotaur.apache.org Received: from mail.apache.org (hermes.apache.org [140.211.11.3]) by minotaur.apache.org (Postfix) with SMTP id 43A8818A39 for ; Wed, 16 Sep 2015 01:44:34 +0000 (UTC) Received: (qmail 39110 invoked by uid 500); 16 Sep 2015 01:44:27 -0000 Delivered-To: apmail-phoenix-user-archive@phoenix.apache.org Received: (qmail 39062 invoked by uid 500); 16 Sep 2015 01:44:27 -0000 Mailing-List: contact user-help@phoenix.apache.org; run by ezmlm Precedence: bulk List-Help: List-Unsubscribe: List-Post: List-Id: Reply-To: user@phoenix.apache.org Delivered-To: mailing list user@phoenix.apache.org Received: (qmail 39053 invoked by uid 99); 16 Sep 2015 01:44:27 -0000 Received: from mail-relay.apache.org (HELO mail-relay.apache.org) (140.211.11.15) by apache.org (qpsmtpd/0.29) with ESMTP; Wed, 16 Sep 2015 01:44:27 +0000 Received: from mail-ig0-f181.google.com (mail-ig0-f181.google.com [209.85.213.181]) by mail-relay.apache.org (ASF Mail Server at mail-relay.apache.org) with ESMTPSA id 8A48E1A0180 for ; Wed, 16 Sep 2015 01:44:27 +0000 (UTC) Received: by igcrk20 with SMTP id rk20so25795271igc.1 for ; Tue, 15 Sep 2015 18:44:26 -0700 (PDT) MIME-Version: 1.0 X-Received: by 10.50.61.230 with SMTP id t6mr12027412igr.54.1442367866470; Tue, 15 Sep 2015 18:44:26 -0700 (PDT) Received: by 10.36.26.135 with HTTP; Tue, 15 Sep 2015 18:44:26 -0700 (PDT) In-Reply-To: References: <55F877DE.4070606@mendeley.com> Date: Tue, 15 Sep 2015 18:44:26 -0700 Message-ID: Subject: Re: timeouts for long queries From: James Taylor To: user Content-Type: multipart/alternative; boundary=047d7bdca44e51136d051fd3704b --047d7bdca44e51136d051fd3704b Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: quoted-printable The other important timeout is Phoenix specific: phoenix.query.timeoutMs. Set this in your hbase-site.xml on the client side to the value in milliseconds for the amount of time you're willing to wait before the query finishes. I might be wrong, but I believe the hbase.rpc.timeout config parameter needs to be set in the hbase-site.xml on the server side (i.e. each region server). On Tue, Sep 15, 2015 at 6:29 PM, Ravi Kiran wrote: > Hi James, > You need to increase the value of hbase.rpc.timeout in hbase-site.xml > on your client end. > http://hbase.apache.org/book.html#trouble.client.lease.exception > > Ravi > > On Tue, Sep 15, 2015 at 12:56 PM, James Heather < > james.heather@mendeley.com> wrote: > >> I'm a bit lost as to what I need to change, and where I need to change >> it, to bump up timeouts for this kind of error: >> >> Caused by: org.apache.phoenix.exception.PhoenixIOException: org.apache.p= hoenix.exception.PhoenixIOException: Failed after attempts=3D36, exceptions= : >> >> Tue Sep 15 18:48:13 UTC 2015, null, java.net.SocketTimeoutException: cal= lTimeout=3D60000, callDuration=3D60304: row '=EF=BF=BD>' on table 'LOADTEST= .TESTING' at region=3DLOADTEST.TESTING,\x03\x00\x00\x00\x00\x00\x00\x00\x00= ,1442332822105.b6b3682074d6c65bd4efa3f1e2b58ffa., hostname=3Dip-172-31-31-1= 77.ec2.chonp.net,60020,1442309899160, seqNum=3D2 >> >> at org.apache.phoenix.util.ServerUtil.parseServerException(Serve= rUtil.java:108) >> >> at org.apache.phoenix.iterate.BaseResultIterators.getIterators(B= aseResultIterators.java:538) >> >> at org.apache.phoenix.iterate.ConcatResultIterator.getIterators(= ConcatResultIterator.java:50) >> >> at org.apache.phoenix.iterate.ConcatResultIterator.currentIterat= or(ConcatResultIterator.java:97) >> >> at org.apache.phoenix.iterate.ConcatResultIterator.next(ConcatRe= sultIterator.java:117) >> >> at org.apache.phoenix.iterate.BaseGroupedAggregatingResultIterat= or.next(BaseGroupedAggregatingResultIterator.java:64) >> >> at org.apache.phoenix.iterate.UngroupedAggregatingResultIterator= .next(UngroupedAggregatingResultIterator.java:39) >> >> at org.apache.phoenix.iterate.DelegateResultIterator.next(Delega= teResultIterator.java:44) >> >> at org.apache.phoenix.iterate.LimitingResultIterator.next(Limiti= ngResultIterator.java:47) >> >> at org.apache.phoenix.jdbc.PhoenixResultSet.next(PhoenixResultSe= t.java:764) >> >> at org.skife.jdbi.v2.Query$4.munge(Query.java:176) >> >> at org.skife.jdbi.v2.QueryResultSetMunger.munge(QueryResultSetMu= nger.java:42) >> >> at org.skife.jdbi.v2.SQLStatement.internalExecute(SQLStatement.j= ava:1340) >> >> ... 20 more >> >> Caused by: java.util.concurrent.ExecutionException: org.apache.phoenix.e= xception.PhoenixIOException: Failed after attempts=3D36, exceptions: >> >> Tue Sep 15 18:48:13 UTC 2015, null, java.net.SocketTimeoutException: cal= lTimeout=3D60000, callDuration=3D60304: row '=EF=BF=BD>' on table 'LOADTEST= .TESTING' at region=3DLOADTEST.TESTING,\x03\x00\x00\x00\x00\x00\x00\x00\x00= ,1442332822105.b6b3682074d6c65bd4efa3f1e2b58ffa., hostname=3Dip-172-31-31-1= 77.ec2.chonp.net,60020,1442309899160, seqNum=3D2 >> >> at java.util.concurrent.FutureTask.report(FutureTask.java:122) >> >> at java.util.concurrent.FutureTask.get(FutureTask.java:206) >> >> at org.apache.phoenix.iterate.BaseResultIterators.getIterators(B= aseResultIterators.java:534) >> >> ... 31 more >> >> >> Is this a client-side timeout, or do I need to change something >> HBase-related on the server and restart the cluster? On master, or all >> region servers? >> >> If it's a client-side thing, where (in JDBC terms) do I do this? >> >> I've tried various things, but I always hit this timeout, and it always >> says the timeout is 60000 (ms, presumably). >> >> James >> > > --047d7bdca44e51136d051fd3704b Content-Type: text/html; charset=UTF-8 Content-Transfer-Encoding: quoted-printable
The other important timeout is Phoenix specific:=C2=A0phoe= nix.query.timeoutMs. Set this in your hbase-site.xml on the client side to = the value in milliseconds for the amount of time you're willing to wait= before the query finishes. I might be wrong, but I believe the=C2=A0hbase.= rpc.timeout config parameter needs to be set in the hbase-site.xml on the s= erver side (i.e. each region server).

<= div class=3D"gmail_quote">On Tue, Sep 15, 2015 at 6:29 PM, Ravi Kiran <maghamravikiran@gmail.com> wrote:
Hi James,
=C2=A0 =C2=A0You need to increase the v= alue of hbase.rpc.timeout in hbase-site.xml on your client end.=C2=A0
=
http://hbase.apache.org/book.html#trouble.client.lease.exception
<= /div>

Ravi

On Tue, Sep 15, 2015 at 12:56 PM, James Hea= ther <james.heather@mendeley.com> wrote:
=20 =20 =20
I'm a bit lost as to what I need to change, and where I need to change it, to bump up timeouts for this kind of error:

Caused by: org.apache.phoenix.exception.PhoenixIOException: org.=
apache.phoenix.exception.PhoenixIOException: Failed after attempts=3D36, ex=
ceptions:
Tue Sep 15 18:48:13 UTC 2015, null, java.net.SocketTimeoutExcept=
ion: callTimeout=3D60000, callDuration=3D60304: row '=EF=BF=BD>'=
 on table 'LOADTEST.TESTING' at region=3DLOADTEST.TESTING,\x03\x00\=
x00\x00\x00\x00\x00\x00\x00,1442332822105.b6b3682074d6c65bd4efa3f1e2b58ffa.=
, hostname=3Dip-172-31-31-177.ec2.chonp.net,60020,1442309899160, seqNum=3D2
      
=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 at org.apache.phoenix=
.util.ServerUtil.parseServerException(ServerUtil.java:108)

      
=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 at org.apache.phoenix=
.iterate.BaseResultIterators.getIterators(BaseResultIterators.java:538)
      
=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 at org.apache.phoenix=
.iterate.ConcatResultIterator.getIterators(ConcatResultIterator.java:50)
      
=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 at org.apache.phoenix=
.iterate.ConcatResultIterator.currentIterator(ConcatResultIterator.java:97)=


      
=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 at org.apache.phoenix=
.iterate.ConcatResultIterator.next(ConcatResultIterator.java:117)

      
=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 at org.apache.phoenix=
.iterate.BaseGroupedAggregatingResultIterator.next(BaseGroupedAggregatingRe=
sultIterator.java:64)

      
=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 at org.apache.phoenix=
.iterate.UngroupedAggregatingResultIterator.next(UngroupedAggregatingResult=
Iterator.java:39)

      
=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 at org.apache.phoenix=
.iterate.DelegateResultIterator.next(DelegateResultIterator.java:44)

      
=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 at org.apache.phoenix=
.iterate.LimitingResultIterator.next(LimitingResultIterator.java:47)

      
=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 at org.apache.phoenix=
.jdbc.PhoenixResultSet.next(PhoenixResultSet.java:764)

      
=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 at org.skife.jdbi.v2.=
Query$4.munge(Query.java:176)

      
=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 at org.skife.jdbi.v2.=
QueryResultSetMunger.munge(QueryResultSetMunger.java:42)

      
=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 at org.skife.jdbi.v2.=
SQLStatement.internalExecute(SQLStatement.java:1340)

      
=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 ... 20 more

      
Caused by: java.util.concurrent.ExecutionException: org.apache.p=
hoenix.exception.PhoenixIOException: Failed after attempts=3D36, exceptions=
:

      
Tue Sep 15 18:48:13 UTC 2015, null, java.net.SocketTimeoutExcept=
ion: callTimeout=3D60000, callDuration=3D60304: row '=EF=BF=BD>'=
 on table 'LOADTEST.TESTING' at region=3DLOADTEST.TESTING,\x03\x00\=
x00\x00\x00\x00\x00\x00\x00,1442332822105.b6b3682074d6c65bd4efa3f1e2b58ffa.=
, hostname=3Dip-172-31-31-177.ec2.chonp.net,60020,1442309899160, seqNum=3D2
      
=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 at java.util.concurre=
nt.FutureTask.report(FutureTask.java:122)

      
=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 at java.util.concurre=
nt.FutureTask.get(FutureTask.java:206)

      
=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 at org.apache.phoenix=
.iterate.BaseResultIterators.getIterators(BaseResultIterators.java:534)
      
=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 ... 31 more

Is this a client-side timeout, or do I need to change something HBase-related on the server and restart the cluster? On master, or all region servers?

If it's a client-side thing, where (in JDBC terms) do I do this?
I've tried various things, but I always hit this timeout, and it always says the timeout is 60000 (ms, presumably).

James


--047d7bdca44e51136d051fd3704b--