phoenix-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "rajeshbabu (JIRA)" <j...@apache.org>
Subject [jira] [Created] (PHOENIX-1149) Index mutations should be committed as batches while building local index
Date Wed, 06 Aug 2014 06:27:12 GMT
rajeshbabu created PHOENIX-1149:
-----------------------------------

             Summary: Index mutations should be committed as batches while building local
index
                 Key: PHOENIX-1149
                 URL: https://issues.apache.org/jira/browse/PHOENIX-1149
             Project: Phoenix
          Issue Type: Bug
            Reporter: rajeshbabu
            Assignee: rajeshbabu
             Fix For: 5.0.0, 4.1


Currently while building local indexes  batchSize is 0 so we are not committing the index
mutations until full region scan is done and which may cause high memory usage or OOM with
huge data. 

{code}
                        // Commit in batches based on UPSERT_BATCH_SIZE_ATTRIB in config
                        if (!indexMutations.isEmpty() && batchSize > 0 &&
indexMutations.size() % batchSize == 0) {
                            HRegion indexRegion = getIndexRegion(c.getEnvironment());
                            // Get indexRegion corresponding to data region
                            commitBatch(indexRegion, indexMutations, null);
                            indexMutations.clear();
                        }
{code}

Thanks for finding this [~mujtabachohan].



--
This message was sent by Atlassian JIRA
(v6.2#6252)

Mime
View raw message