hive-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "ASF GitHub Bot (JIRA)" <j...@apache.org>
Subject [jira] [Work logged] (HIVE-19430) ObjectStore.cleanNotificationEvents OutOfMemory on large number of pending events
Date Fri, 15 Feb 2019 11:43:00 GMT

     [ https://issues.apache.org/jira/browse/HIVE-19430?focusedWorklogId=199175&page=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-199175
]

ASF GitHub Bot logged work on HIVE-19430:
-----------------------------------------

                Author: ASF GitHub Bot
            Created on: 15/Feb/19 11:42
            Start Date: 15/Feb/19 11:42
    Worklog Time Spent: 10m 
      Work Description: sankarh commented on pull request #537: HIVE-19430 : ObjectStore.cleanNotificationEvents
OutOfMemory on large number of pending events
URL: https://github.com/apache/hive/pull/537#discussion_r257201879
 
 

 ##########
 File path: standalone-metastore/metastore-common/src/main/java/org/apache/hadoop/hive/metastore/conf/MetastoreConf.java
 ##########
 @@ -555,6 +555,9 @@ public static ConfVars getMetaConf(String name) {
     EVENT_DB_LISTENER_TTL("metastore.event.db.listener.timetolive",
         "hive.metastore.event.db.listener.timetolive", 86400, TimeUnit.SECONDS,
         "time after which events will be removed from the database listener queue"),
+    CLEAN_MAX_EVENTS("metastore.event.db.clean.maxevents",
 
 Review comment:
   EVENT_ prefix is used for other related configs. Can use the same here as well.
 
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
users@infra.apache.org


Issue Time Tracking
-------------------

    Worklog Id:     (was: 199175)
    Time Spent: 40m  (was: 0.5h)

> ObjectStore.cleanNotificationEvents OutOfMemory on large number of pending events
> ---------------------------------------------------------------------------------
>
>                 Key: HIVE-19430
>                 URL: https://issues.apache.org/jira/browse/HIVE-19430
>             Project: Hive
>          Issue Type: Bug
>            Reporter: Thejas M Nair
>            Assignee: Ashutosh Bapat
>            Priority: Major
>              Labels: pull-request-available
>             Fix For: 4.0.0
>
>         Attachments: HIVE-19430.01.patch
>
>          Time Spent: 40m
>  Remaining Estimate: 0h
>
> If there are large number of events that haven't been cleaned up for some reason, then
ObjectStore.cleanNotificationEvents() can run out of memory while it loads all the events
to be deleted.
> It should fetch events in batches.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

Mime
View raw message