hadoop-common-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Colin Patrick McCabe (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (HADOOP-10027) *Compressor_deflateBytesDirect passes instance instead of jclass to GetStaticObjectField
Date Mon, 02 Mar 2015 22:16:06 GMT

    [ https://issues.apache.org/jira/browse/HADOOP-10027?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14343874#comment-14343874

Colin Patrick McCabe commented on HADOOP-10027:

bq. Chris wrote: My best friend, the git blame log, tells me that this traces back to commit
9e22afd8704f62e312e749ebdb882455569de560, which was HADOOP-3604 from 2008. We should be able
to use the information in that issue to trace back to specific JVM versions that exhibited
this bug. Assuming those are all very old JVM versions, then I am +1 for the approach of removing
this locking completely. Thanks for taking this on Hui Zheng.

Thanks, Chris.  Very good research.  Indeed, the "fix version" on all those java bugs is somewhere
between 1.3 and 1.5, inclusive.  Since our min JDK version is 1.7 now (or possibly 1.6 if
we backport this to 2.6.x), it should be an issue.

I agree that the new {{testZlibCompressDecompressInMultiThreads}} function needs some work.
 We definitely should be joining those threads before the end of the test.  Does it make sense
to commit what we have now (minus the new unit test), and create a new JIRA for adding proper
bzip2 test coverage?  That might keep the scope more manageable.  I am certainly +1 on the
non-unit-test part of the patch as-is.

> *Compressor_deflateBytesDirect passes instance instead of jclass to GetStaticObjectField
> ----------------------------------------------------------------------------------------
>                 Key: HADOOP-10027
>                 URL: https://issues.apache.org/jira/browse/HADOOP-10027
>             Project: Hadoop Common
>          Issue Type: Bug
>          Components: native
>            Reporter: Eric Abbott
>            Assignee: Hui Zheng
>            Priority: Minor
>         Attachments: HADOOP-10027.1.patch, HADOOP-10027.2.patch
> http://svn.apache.org/viewvc/hadoop/common/trunk/hadoop-common-project/hadoop-common/src/main/native/src/org/apache/hadoop/io/compress/zlib/ZlibCompressor.c?view=markup
> This pattern appears in all the native compressors.
>     // Get members of ZlibCompressor
>     jobject clazz = (*env)->GetStaticObjectField(env, this,
>                                                  ZlibCompressor_clazz);
> The 2nd argument to GetStaticObjectField is supposed to be a jclass, not a jobject. Adding
the JVM param -Xcheck:jni will cause "FATAL ERROR in native method: JNI received a class argument
that is not a class" and a core dump such as the following.
> (gdb) 
> #0 0x00007f02e4aef8a5 in raise () from /lib64/libc.so.6
> #1 0x00007f02e4af1085 in abort () from /lib64/libc.so.6
> #2 0x00007f02e45bd727 in os::abort(bool) () from /opt/jdk1.6.0_31/jre/lib/amd64/server/libjvm.so
> #3 0x00007f02e43cec63 in jniCheck::validate_class(JavaThread*, _jclass*, bool) () from
> #4 0x00007f02e43ea669 in checked_jni_GetStaticObjectField () from /opt/jdk1.6.0_31/jre/lib/amd64/server/libjvm.so
> #5 0x00007f02d38eaf79 in Java_org_apache_hadoop_io_compress_zlib_ZlibCompressor_deflateBytesDirect
() from /usr/lib/hadoop/lib/native/libhadoop.so.1.0.0
> In addition, that clazz object is only used for synchronization. In the case of the native
method _deflateBytesDirect, the result is a class wide lock used to access the instance field
uncompressed_direct_buf. Perhaps using the instance as the sync point is more appropriate?

This message was sent by Atlassian JIRA

View raw message