phoenix-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Vincent Poon (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (PHOENIX-4683) Cap timeouts for stats precompact hook logic
Date Tue, 03 Apr 2018 00:05:00 GMT

    [ https://issues.apache.org/jira/browse/PHOENIX-4683?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16423306#comment-16423306
] 

Vincent Poon commented on PHOENIX-4683:
---------------------------------------

[~jamestaylor] when testing this, I'm still seeing a long timeout in effect, I think because
in DefaultStatisticsCollector#initGuidePostDepth() , we are using the env.getTable() , which
I guess has the environment's config.  Know of any workaround?

2018-04-03 00:02:25,751 INFO  [ctions-1522712936039] client.RpcRetryingCaller - Call exception,
tries=35, retries=350, retryTime=530682ms, msg=row '^@^@RESTORE_AA_RELEASEVAL1' on table 'SYSTEM.CATALOG'
at region=SYSTEM.CATALOG,1522231405872.61e33182ecf40e43dcbeb25414922c6b., hostname=stg2hdaas-dnds1-2-prd.eng.sfdc.net,60020,1522712870583,
seqNum=693

"regionserver60020-smallCompactions-1522712936288" #1041 daemon prio=5 os_prio=0 tid=0x00007fa9949aa000
nid=0x16c88 waiting on condition [0x00007fa0f7478000]
 java.lang.Thread.State: TIMED_WAITING (sleeping)
 at java.lang.Thread.sleep(Native Method)
 at org.apache.hadoop.hbase.client.RpcRetryingCaller.callWithRetries(RpcRetryingCaller.java:159)
 - locked <0x00007fa973377200> (a org.apache.hadoop.hbase.client.RpcRetryingCaller)
 at org.apache.hadoop.hbase.client.HTable.get(HTable.java:842)
 at org.apache.hadoop.hbase.client.HTable.get(HTable.java:819)
 at org.apache.hadoop.hbase.coprocessor.CoprocessorHost$Environment$HTableWrapper.get(CoprocessorHost.java:487)
 at org.apache.phoenix.schema.stats.DefaultStatisticsCollector.initGuidepostDepth(DefaultStatisticsCollector.java:147)
 at org.apache.phoenix.schema.stats.DefaultStatisticsCollector.init(DefaultStatisticsCollector.java:367)
 at org.apache.phoenix.schema.stats.DefaultStatisticsCollector.createCompactionScanner(DefaultStatisticsCollector.java:359)
 at org.apache.phoenix.coprocessor.UngroupedAggregateRegionObserver$2.run(UngroupedAggregateRegionObserver.java:910)
 at org.apache.phoenix.coprocessor.UngroupedAggregateRegionObserver$2.run(UngroupedAggregateRegionObserver.java:900)
 at java.security.AccessController.doPrivileged(Native Method)
 at javax.security.auth.Subject.doAs(Subject.java:422)
 at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1760)
 at org.apache.phoenix.coprocessor.UngroupedAggregateRegionObserver.preCompact(UngroupedAggregateRegionObserver.java:900)
 at org.apache.hadoop.hbase.coprocessor.BaseRegionObserver.preCompact(BaseRegionObserver.java:197)

> Cap timeouts for stats precompact hook logic
> --------------------------------------------
>
>                 Key: PHOENIX-4683
>                 URL: https://issues.apache.org/jira/browse/PHOENIX-4683
>             Project: Phoenix
>          Issue Type: Bug
>    Affects Versions: 4.13.0
>            Reporter: Vincent Poon
>            Assignee: Vincent Poon
>            Priority: Major
>         Attachments: PHOENIX-4683.v1.0.98.patch, PHOENIX-4683.v2.0.98.patch, PHOENIX-4683.v3.0.98.patch
>
>
> In UngroupedAggregateRegionObserver#preCompact we call DefaultStatisticsCollector.createCompactionScanner. 
It uses the env config which in turn contains the RS server rpc timeout of 20 minutes.  That's
too long for a compaction hook.
> Like in PHOENIX-4169, we should cap the timeout so the compaction doesn't get blocked.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

Mime
View raw message