phoenix-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "rajeshbabu (JIRA)" <j...@apache.org>
Subject [jira] [Updated] (PHOENIX-1149) Index mutations should be committed as batches while building local index
Date Wed, 06 Aug 2014 06:37:11 GMT

     [ https://issues.apache.org/jira/browse/PHOENIX-1149?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]

rajeshbabu updated PHOENIX-1149:
--------------------------------

    Attachment: PHOENIX-1149.patch

Here is the simple patch setting batchSize to default or configured value while building local
index.

> Index mutations should be committed as batches while building local index
> -------------------------------------------------------------------------
>
>                 Key: PHOENIX-1149
>                 URL: https://issues.apache.org/jira/browse/PHOENIX-1149
>             Project: Phoenix
>          Issue Type: Bug
>            Reporter: rajeshbabu
>            Assignee: rajeshbabu
>             Fix For: 5.0.0, 4.1
>
>         Attachments: PHOENIX-1149.patch
>
>
> Currently while building local indexes  batchSize is 0 so we are not committing the index
mutations until full region scan is done and which may cause high memory usage or OOM with
huge data. 
> {code}
>                         // Commit in batches based on UPSERT_BATCH_SIZE_ATTRIB in config
>                         if (!indexMutations.isEmpty() && batchSize > 0 &&
indexMutations.size() % batchSize == 0) {
>                             HRegion indexRegion = getIndexRegion(c.getEnvironment());
>                             // Get indexRegion corresponding to data region
>                             commitBatch(indexRegion, indexMutations, null);
>                             indexMutations.clear();
>                         }
> {code}
> Thanks for finding this [~mujtabachohan].



--
This message was sent by Atlassian JIRA
(v6.2#6252)

Mime
View raw message