hadoop-common-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Kai Zheng (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (HADOOP-11541) Raw XOR coder
Date Fri, 06 Feb 2015 01:49:38 GMT

    [ https://issues.apache.org/jira/browse/HADOOP-11541?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14308438#comment-14308438

Kai Zheng commented on HADOOP-11541:

Hi [~zhz],

Thanks for your comments. 
bq.What's the downside of converting to a unified format first?
I'm thinking it's doing the real calculating and performance concerned, so I don't have much
trick here. When we have more experience for implementing the following more raw coders, I
guess we can have some idea about how to unifying them.
bq.TestCoderBase needs better documentation
Agree, will do. 
bq.What does {{getErasedIndexesForDecoding}} do? What is {{erasedDataIndexes}} for?
To test recovering, we intentively erase some data units specified by {{erasedDataIndexes}}.
{{getErasedIndexesForDecoding}} is to adjust the indexes to pass to raw coder to invoke the
decode() function. Will have more comments for them.
bq.{{testCoding}} can be in {{TestCoderBase}}
{{TestCoderBase}} is not only for tests of raw coders, but also will be used for tests of
block coders, which can be seen in the patch for HDFS-7662. Therefore I don't put raw coder
specific codes in the base class.

> Raw XOR coder
> -------------
>                 Key: HADOOP-11541
>                 URL: https://issues.apache.org/jira/browse/HADOOP-11541
>             Project: Hadoop Common
>          Issue Type: Sub-task
>            Reporter: Kai Zheng
>            Assignee: Kai Zheng
>             Fix For: HDFS-EC
>         Attachments: HADOOP-11541-v1.patch
> This will implement XOR codes by porting the codes from HDFS-RAID. The coder in the algorithm
is needed by some high level codecs like LRC.

This message was sent by Atlassian JIRA

View raw message