hadoop-mapreduce-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "MacChen01 (Jira)" <j...@apache.org>
Subject [jira] MacChen01 shared "HADOOP-16601: Add support for hardware crc32 of nativetask checksums on aarch64 arch" with you
Date Thu, 17 Oct 2019 02:45:00 GMT
    MacChen01 shared an issue with you
    ------------------------------------

    This my first patch for hadoop, please 
 help me review the patch. What should I do next?

> Add support for hardware crc32 of nativetask checksums on aarch64 arch
> ----------------------------------------------------------------------
>
>                 Key: HADOOP-16601
>                 URL: https://issues.apache.org/jira/browse/HADOOP-16601
>             Project: Hadoop Common
>          Issue Type: Improvement
>          Components: common
>    Affects Versions: 3.2.1
>            Reporter: MacChen01
>            Priority: Major
>              Labels: performance
>             Fix For: site
>
>         Attachments: HADOOP-16601.patch
>
>
> Add support for aarch64 CRC instructions in nativetask module, optimize the CRC32 and
CRC32C.
> Use the benchmark tools : nttest , the improvement is quite substantial:  
> *CRC32 Zlib polynomial 0x04C11DB7*
> |KeyValueType-IO|Before(MB/s)|After(MB/s)|Improvement|
> |TextType-Write|425.98|602.92|+42%|
> |TextType-Read|796.06|1716.59|+116%|
> |BytesType-Write|474.25|686.84|+45%|
> |BytesType-Read|844.96|1955.03|+131%|
> |UnknownType-Write|434.84|608.81|+40%|
> |UnknownType-Read|805.76|1733.82|+115%|
>  
>   
>  *CRC32C  Castagnoli polynomial 0x1EDC6F41*
>  
> |KeyValueType-IO|Before(MB/s)|After(MB/s)|Improvement|
> |TextType-Write|423.39|606.55|+43%|
> |TextType-Read|799.20|1783.28|+123%|
> |BytesType-Write|473.95|696.47|+47%|
> |BytesType-Read|846.30|2018.06|+138%|
> |UnknownType-Write|434.07|612.31|+41%|
> |UnknownType-Read|807.16|1783.95|+121%|

     Also shared with
          common-dev@hadoop.apache.org



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

---------------------------------------------------------------------
To unsubscribe, e-mail: mapreduce-dev-unsubscribe@hadoop.apache.org
For additional commands, e-mail: mapreduce-dev-help@hadoop.apache.org


Mime
View raw message