hadoop-common-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Kai Zheng (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (HADOOP-13010) Refactor raw erasure coders
Date Thu, 19 May 2016 13:43:12 GMT

    [ https://issues.apache.org/jira/browse/HADOOP-13010?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15291102#comment-15291102

Kai Zheng commented on HADOOP-13010:

bq. It seems better just to have one function, createRawEncoder, than to have lots of functions
for every type of encoder.
Ok let's just have one function. Based on the new approach you suggested, we may be clear
then how many lines of codes would be needed to create the primitive rs coder, and if it's
rather simple ok no shortcut alias method is needed, otherwise it would sound good to have
the shortcut method because we have many places in HDFS side to create the coder, avoiding
duplication of lines of codes. Anyway let we rebase the codes first trying to use the ONE
bq. Reducing the number of function parameters from 8 or 9 to 1 or 2 seems like it would make
the code much more readable. I don't understand what the rationale is for keeping these parameters
out of DecodingState. Perhaps we could discuss this in a follow-on JIRA, though.
I agree it's doable if we would reorganize the codes some bit because in my rational, if you
move the variables to the class, then the relevant codes using the variables would also be
moved as well, otherwise the class of DecodingState would be just like a c-style struct of
some fields. The other reason I mentioned is DecodingState in the direct bytebuffer case wouldn't
need these variables because they're just for bytes array backed buffers.

> Refactor raw erasure coders
> ---------------------------
>                 Key: HADOOP-13010
>                 URL: https://issues.apache.org/jira/browse/HADOOP-13010
>             Project: Hadoop Common
>          Issue Type: Sub-task
>            Reporter: Kai Zheng
>            Assignee: Kai Zheng
>         Attachments: HADOOP-13010-v1.patch, HADOOP-13010-v2.patch, HADOOP-13010-v3.patch,
HADOOP-13010-v4.patch, HADOOP-13010-v5.patch
> This will refactor raw erasure coders according to some comments received so far.
> * As discussed in HADOOP-11540 and suggested by [~cmccabe], better not to rely class
inheritance to reuse the codes, instead they can be moved to some utility.
> * Suggested by [~jingzhao] somewhere quite some time ago, better to have a state holder
to keep some checking results for later reuse during an encode/decode call.
> This would not get rid of some inheritance levels as doing so isn't clear yet for the
moment and also incurs big impact. I do wish the end result by this refactoring will make
all the levels more clear and easier to follow.

This message was sent by Atlassian JIRA

To unsubscribe, e-mail: common-issues-unsubscribe@hadoop.apache.org
For additional commands, e-mail: common-issues-help@hadoop.apache.org

View raw message