hive-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "mahesh kumar behera (JIRA)" <j...@apache.org>
Subject [jira] [Updated] (HIVE-20511) REPL DUMP is leaking metastore connections
Date Fri, 21 Sep 2018 09:39:00 GMT

     [ https://issues.apache.org/jira/browse/HIVE-20511?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]

mahesh kumar behera updated HIVE-20511:
---------------------------------------
    Status: Open  (was: Patch Available)

> REPL DUMP is leaking metastore connections
> ------------------------------------------
>
>                 Key: HIVE-20511
>                 URL: https://issues.apache.org/jira/browse/HIVE-20511
>             Project: Hive
>          Issue Type: Bug
>          Components: repl
>    Affects Versions: 4.0.0
>            Reporter: mahesh kumar behera
>            Assignee: mahesh kumar behera
>            Priority: Major
>              Labels: pull-request-available
>             Fix For: 4.0.0
>
>         Attachments: HIVE-20511.01.patch, HIVE-20511.02.patch, HIVE-20511.03.patch, HIVE-20511.04.patch,
HIVE-20511.05.patch
>
>
> With remote metastore, REPL DUMP  leaking connections. Each repl dump task is leaking
one connection due to the usage of stale hive object. 
> {code}
> 18/09/04 16:01:46 INFO ReplState: REPL::EVENT_DUMP: {"dbName":"*","eventId":"566","eventType":"EVENT_COMMIT_TXN","eventsDumpProgress":"1/0","dumpTime":1536076906}
> 18/09/04 16:01:46 INFO events.AbstractEventHandler: Processing#567 OPEN_TXN message :
{"txnIds":null,"timestamp":1536076905,"fromTxnId":269,"toTxnId":269,"server":"thrift://metastore-service.warehouse-1536062326-s74h.svc.cluster.local:9083","servicePrincipal":""}
> 18/09/04 16:01:46 INFO ReplState: REPL::EVENT_DUMP: {"dbName":"*","eventId":"567","eventType":"EVENT_OPEN_TXN","eventsDumpProgress":"2/0","dumpTime":1536076906}
> 18/09/04 16:01:46 INFO metastore.HiveMetaStoreClient: Trying to connect to metastore
with URI thrift://metastore-service.warehouse-1536062326-s74h.svc.cluster.local:9083
> 18/09/04 16:01:46 INFO metastore.HiveMetaStoreClient: Opened a connection to metastore,
current connections: 471
> 18/09/04 16:01:46 INFO metastore.HiveMetaStoreClient: Connected to metastore.
> 18/09/04 16:01:46 INFO metastore.RetryingMetaStoreClient: RetryingMetaStoreClient proxy=class
org.apache.hadoop.hive.ql.metadata.SessionHiveMetaStoreClient ugi=hive (auth:SIMPLE) retries=24
delay=5 lifetime=0
> 18/09/04 16:01:46 INFO ReplState: REPL::END: {"dbName":"*","dumpType":"INCREMENTAL","actualNumEvents":2,"dumpEndTime":1536076906,"dumpDir":"/user/hive/repl/e45bde27-74dc-45cd-9823-400a8fc1aea3","lastReplId":"567"}
> 18/09/04 16:01:46 INFO repl.ReplDumpTask: Done dumping events, preparing to return /user/hive/repl/e45bde27-74dc-45cd-9823-400a8fc1aea3,567
> 18/09/04 16:01:46 INFO ql.Driver: Completed executing command(queryId=hive_20180904160145_30f9570a-44e0-4f3b-b961-1906d3972fc4);
Time taken: 0.585 seconds
> OK
> 18/09/04 16:01:46 INFO ql.Driver: OK
> 18/09/04 16:01:46 INFO lockmgr.DbTxnManager: Stopped heartbeat for query: hive_20180904160145_30f9570a-44e0-4f3b-b961-1906d3972fc4
> 18/09/04 16:01:46 INFO metastore.HiveMetaStoreClient: Trying to connect to metastore
with URI thrift://metastore-service.warehouse-1536062326-s74h.svc.cluster.local:9083
> 18/09/04 16:01:46 INFO metastore.HiveMetaStoreClient: Opened a connection to metastore,
current connections: 472
> 18/09/04 16:01:46 INFO metastore.HiveMetaStoreClient: Connected to metastore.
> {code}



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

Mime
View raw message