phoenix-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Hudson (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (PHOENIX-1149) Index mutations should be committed as batches while building local index
Date Wed, 06 Aug 2014 17:24:12 GMT

    [ https://issues.apache.org/jira/browse/PHOENIX-1149?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14087917#comment-14087917
] 

Hudson commented on PHOENIX-1149:
---------------------------------

FAILURE: Integrated in Phoenix | Master | Hadoop1 #318 (See [https://builds.apache.org/job/Phoenix-master-hadoop1/318/])
PHOENIX-1149 Index mutations should be committed as batches while building local index (Rajeshbabu)
(jtaylor: rev 0d8c04cf8061bf658afab0241453fea662c1d22b)
* phoenix-core/src/main/java/org/apache/phoenix/coprocessor/UngroupedAggregateRegionObserver.java
* phoenix-core/src/main/java/org/apache/phoenix/compile/WhereOptimizer.java


> Index mutations should be committed as batches while building local index
> -------------------------------------------------------------------------
>
>                 Key: PHOENIX-1149
>                 URL: https://issues.apache.org/jira/browse/PHOENIX-1149
>             Project: Phoenix
>          Issue Type: Bug
>            Reporter: rajeshbabu
>            Assignee: rajeshbabu
>             Fix For: 5.0.0, 4.1
>
>         Attachments: PHOENIX-1149.patch
>
>
> Currently while building local indexes  batchSize is 0 so we are not committing the index
mutations until full region scan is done and which may cause high memory usage or OOM with
huge data. 
> {code}
>                         // Commit in batches based on UPSERT_BATCH_SIZE_ATTRIB in config
>                         if (!indexMutations.isEmpty() && batchSize > 0 &&
indexMutations.size() % batchSize == 0) {
>                             HRegion indexRegion = getIndexRegion(c.getEnvironment());
>                             // Get indexRegion corresponding to data region
>                             commitBatch(indexRegion, indexMutations, null);
>                             indexMutations.clear();
>                         }
> {code}
> Thanks for finding this [~mujtabachohan].



--
This message was sent by Atlassian JIRA
(v6.2#6252)

Mime
View raw message