phoenix-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "ASF GitHub Bot (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (PHOENIX-900) Partial results for mutations
Date Tue, 17 Feb 2015 01:05:11 GMT

    [ https://issues.apache.org/jira/browse/PHOENIX-900?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14323517#comment-14323517
] 

ASF GitHub Bot commented on PHOENIX-900:
----------------------------------------

Github user JamesRTaylor commented on a diff in the pull request:

    https://github.com/apache/phoenix/pull/37#discussion_r24786114
  
    --- Diff: phoenix-core/src/main/java/org/apache/phoenix/execute/MutationState.java ---
    @@ -488,7 +490,55 @@ public void rollback(PhoenixConnection connection) throws SQLException
{
             numRows = 0;
         }
         
    +    private Set<Integer> getOrderOfUncommittedStatements() {
    +    	Set<Integer> result = newHashSet();
    +    	for (Map<ImmutableBytesPtr, RowMutationState> rowMutations : mutations.values())
{
    +    		for (RowMutationState rowMutationState : rowMutations.values()) {
    +    			result.addAll(rowMutationState.getOrderOfStatementsInConnection());
    +    		}
    +    	}
    +    	return result;
    +    }
    +    
         @Override
         public void close() throws SQLException {
         }
    +    
    +    public static class RowMutationState {
    +    	private Map<PColumn,byte[]> columnValues;
    +    	private Set<Integer> orderOfStatementsInConnection;
    --- End diff --
    
    I'm somewhat worried about the memory bloat of adding another Set object here. An alternative
would be a long[] that acts as a bit set where any set bit represents the statement index.
It's unlikely that there'd be more than 64 statements in a commit, but if there were, you
could always re-allocate the long[]. You could then materialize the set of statement indexes
from it. 
    
    Another more flexible alternative would be to make RowMutationState an interface and have
a SingleStatementRowMutationState that has a single int index member variable (likely the
common case). Then, add a join method that returns a new MultiRowMutationState with a long
as a bitset and maybe one more with a long[] as a bitset.


> Partial results for mutations
> -----------------------------
>
>                 Key: PHOENIX-900
>                 URL: https://issues.apache.org/jira/browse/PHOENIX-900
>             Project: Phoenix
>          Issue Type: Bug
>    Affects Versions: 3.0.0, 4.0.0
>            Reporter: Eli Levine
>            Assignee: Eli Levine
>         Attachments: PHOENIX-900.patch
>
>
> HBase provides a way to retrieve partial results of a batch operation: http://hbase.apache.org/apidocs/org/apache/hadoop/hbase/client/HTable.html#batch%28java.util.List,%20java.lang.Object[]%29
> Chatted with James about this offline:
> Yes, this could be included in the CommitException we throw (MutationState:412). We already
include the batches that have been successfully committed to the HBase server in this exception.
Would you be up for adding this additional information? You'd want to surface this in a Phoenix-y
way in a method on CommitException, something like this: ResultSet getPartialCommits(). You
can easily create an in memory ResultSet using MaterializedResultIterator plus the PhoenixResultSet
constructor that accepts this (just create a new empty PhoenixStatement with the PhoenixConnection
for the other arg).



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

Mime
View raw message