gora-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Ferdy Galema (Commented) (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (GORA-117) gora hbase does not have a mechanism to set the caching on a scanner, which makes for poor performance on map/reduce jobs
Date Wed, 11 Apr 2012 08:06:36 GMT

    [ https://issues.apache.org/jira/browse/GORA-117?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13251407#comment-13251407
] 

Ferdy Galema commented on GORA-117:
-----------------------------------

Never mind the comment about hbase.client.scanner.caching property. (Of course this is something
you already know.)  But it's not fully clear to me yet why this property has no effect in
the current store implementation.

Thanks for raising this issue. I'll await Stack's suggestions.
                
> gora hbase does not have a mechanism to set the caching on a scanner, which makes for
poor performance on map/reduce jobs
> -------------------------------------------------------------------------------------------------------------------------
>
>                 Key: GORA-117
>                 URL: https://issues.apache.org/jira/browse/GORA-117
>             Project: Apache Gora
>          Issue Type: Bug
>          Components: storage-hbase
>            Reporter: Eric Newton
>            Assignee: stack
>
> goraci runs a map/reduce job over all the data that it generates.  The hbase storage
uses a scanner that doesn't cache rows, which means every fetch requires an RPC call.  I experimented
with 
> scan.setCaching(1000);
> and goraci Verify ran about 30x faster.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

        

Mime
View raw message