hadoop-common-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Jason Lowe (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (HADOOP-8462) Native-code implementation of bzip2 codec
Date Wed, 06 Mar 2013 03:04:15 GMT

    [ https://issues.apache.org/jira/browse/HADOOP-8462?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13594280#comment-13594280

Jason Lowe commented on HADOOP-8462:

The trunk patch looks good to me.  The test complaints are spurious, as it gets confused when
whitespace is changed on the line just before a {{Test}} decorator.

I was not able to get the branch-0.23 patch to work.  It's always missing the native support
in libhadoop when I build, even when the bzip2-devel package is present.  Looking at the top
of the C code files:


#include <config.h>

{{HADOOP_BZIP2_LIBRARY}} is provided by {{config.h}} so it appears the {{#if defined}} line
is too early in the file and therefore the native code is always skipped by the preprocessor.
 The code needs to look something like this:

#if defined HAVE_CONFIG_H
  #include <config.h>


> Native-code implementation of bzip2 codec
> -----------------------------------------
>                 Key: HADOOP-8462
>                 URL: https://issues.apache.org/jira/browse/HADOOP-8462
>             Project: Hadoop Common
>          Issue Type: Improvement
>          Components: io
>    Affects Versions: 0.23.1
>            Reporter: Govind Kamat
>            Assignee: Govind Kamat
>         Attachments: HADOOP-8462-0.23.patch, HADOOP-8462-0.23.patch, HADOOP-8462-2.0.2a.1.patch,
HADOOP-8462-2.0.2a.patch, HADOOP-8462-2.0.2a.patch, HADOOP-8462-2.0.2a.patch, HADOOP-8462-trunk.1.patch,
HADOOP-8462-trunk.1.patch, HADOOP-8462-trunk.patch, HADOOP-8462-trunk.patch, HADOOP-8462-trunk.patch,
HADOOP-8462-trunk.patch, HADOOP-8462-trunk.patch, HADOOP-8462-trunk.patch, HADOOP-8462-trunk.patch
>   Original Estimate: 672h
>  Remaining Estimate: 672h
> The bzip2 codec supplied with Hadoop is currently available only as a Java implementation.
 A version that uses the system bzip2 library can provide improved performance and a better
memory footprint.  This will also make it feasible to utilize alternative bzip2 libraries
that may perform better for specific jobs.

This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

View raw message