hadoop-common-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Akira AJISAKA (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (HADOOP-11627) Remove io.native.lib.available from trunk
Date Fri, 03 Apr 2015 12:12:53 GMT

    [ https://issues.apache.org/jira/browse/HADOOP-11627?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14394350#comment-14394350
] 

Akira AJISAKA commented on HADOOP-11627:
----------------------------------------

Thanks [~brahmareddy] for updating the patch. Some comments.
1. Would you remove unused imports from the following files?
* ZlibFactory.java
* BZip2Factory.java
* NativeCodeLoader.java
* TestZlibCompressorDecompressor.java
* TestConcatenatedCompressedInput.java

2. In {{TestCodec.java}}, would you add {{ZlibFactory.setNativeZlibLoaded(false)}} in the
below tests?
{code}
@@ -458,7 +461,6 @@ public void testCodecInitWithCompressionLevel() throws Exception {
                + ": native libs not loaded");
     }
     conf = new Configuration();
-    conf.setBoolean(CommonConfigurationKeys.IO_NATIVE_LIB_AVAILABLE_KEY, false);
     codecTestWithNOCompression( conf,
                          "org.apache.hadoop.io.compress.DefaultCodec");
   }
{code}
{code}
     } else {
       LOG.warn("testCodecPoolCompressorReinit skipped: native libs not loaded");
     }
-    conf.setBoolean(CommonConfigurationKeys.IO_NATIVE_LIB_AVAILABLE_KEY, false);
     DefaultCodec dfc = ReflectionUtils.newInstance(DefaultCodec.class, conf);
     gzipReinitTest(conf, dfc);
   }
{code}
{code}
@@ -901,8 +899,6 @@ public void testCodecPoolAndGzipDecompressor() {
 
     // Don't use native libs for this test.
     Configuration conf = new Configuration();
-    conf.setBoolean(CommonConfigurationKeys.IO_NATIVE_LIB_AVAILABLE_KEY,
-                    false);
{code}

3. I'm thinking try-finally statement in {{TestConcatenatedCompressedInput#testPrototypeInflaterGzip()}}
 is not needed. Adding {{ZlibFactory.loadNativeZLib()}} before {{doMultipleGzipBufferSizes(jobConf,
true)}} is sufficient for me.
{code}
@@ -352,7 +360,13 @@ public void testBuiltInGzipDecompressor() throws IOException {
                  84, lineNum);
 
     // test BuiltInGzipDecompressor with lots of different input-buffer sizes
-    doMultipleGzipBufferSizes(jobConf, false);
+    // Don't use native libs
+    try {
+      ZlibFactory.setNativeZlibLoaded(false);
+      doMultipleGzipBufferSizes(jobConf, false);
+    } finally {
+      ZlibFactory.loadNativeZLib();
+    }
{code}

> Remove io.native.lib.available from trunk
> -----------------------------------------
>
>                 Key: HADOOP-11627
>                 URL: https://issues.apache.org/jira/browse/HADOOP-11627
>             Project: Hadoop Common
>          Issue Type: Improvement
>    Affects Versions: 3.0.0
>            Reporter: Akira AJISAKA
>            Assignee: Brahma Reddy Battula
>         Attachments: HADOOP-11627-002.patch, HADOOP-11627-003.patch, HADOOP-11627-004.patch,
HADOOP-11627-005.patch, HADOOP-11627-006.patch, HADOOP-11627.patch
>
>
> According to the discussion in HADOOP-8642, we should remove {{io.native.lib.available}}
from trunk, and always use native libraries if they exist.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

Mime
View raw message