hadoop-mapreduce-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Wing Yew Poon (Created) (JIRA)" <j...@apache.org>
Subject [jira] [Created] (MAPREDUCE-3515) hadoop 0.23: native compression libraries not being loaded
Date Tue, 06 Dec 2011 22:48:39 GMT
hadoop 0.23: native compression libraries not being loaded

                 Key: MAPREDUCE-3515
                 URL: https://issues.apache.org/jira/browse/MAPREDUCE-3515
             Project: Hadoop Map/Reduce
          Issue Type: Bug
          Components: mrv2
    Affects Versions: 0.23.0
            Reporter: Wing Yew Poon

I installed the hadoop package from the Bigtop hadoop 0.23 branch. Among other files, the
package installs


I ran a simple job using compression:

hadoop jar /usr/lib/hadoop/hadoop-mapreduce-examples.jar wordcount -D mapreduce.output.fileoutputformat.compress=true
-D mapreduce.output.fileoutputformat.compress.codec=org.apache.hadoop.io.compress.GzipCodec
examples/text wordcount-gz

I see

11/12/06 13:42:06 WARN util.NativeCodeLoader: Unable to load native-hadoop library for your
platform... using builtin-java classes where applicable
11/12/06 13:42:06 WARN snappy.LoadSnappy: Snappy native library not loaded

and at the end, I see

-rw-r--r--   1 root supergroup          0 2011-12-06 13:42 wordcount-gz/_SUCCESS
-rw-r--r--   1 root supergroup      46228 2011-12-06 13:42 wordcount-gz/part-r-00000.gz

so the output is compressed, but from the log message, I assume that the native library is
not being loaded and that the java gzip is being used.

This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira


View raw message