build 148 broke the unmap hack. The problem is an additional bug in Java: The unmap hack should be more unspecific on the Exception Type (InaccessibleObjectException does not extend ReflectiveOperationException). I will change this in an issue, but the general problem persists:


With the buil 148 version we can no longer unmap byte buffers:


java.lang.reflect.InaccessibleObjectException: Unable to make public jdk.internal.ref.Cleaner java.nio.DirectByteBuffer.cleaner() accessible: module java.base does not "opens java.nio" to unnamed module @10b73cf

   [junit4]    >       at java.base/jdk.internal.reflect.Reflection.throwInaccessibleObjectException(Reflection.java:427)

   [junit4]    >       at java.base/java.lang.reflect.AccessibleObject.checkCanSetAccessible(AccessibleObject.java:201)

   [junit4]    >       at java.base/java.lang.reflect.Method.checkCanSetAccessible(Method.java:192)

   [junit4]    >       at java.base/java.lang.reflect.Method.setAccessible(Method.java:186)

   [junit4]    >       at org.apache.lucene.store.MMapDirectory.unmapHackImpl(MMapDirectory.java:345)

   [junit4]    >       at java.base/java.security.AccessController.doPrivileged(Native Method)

   [junit4]    >       at org.apache.lucene.store.MMapDirectory.<clinit>(MMapDirectory.java:326)

   [junit4]    >       ... 42 moreThrowable #2: java.lang.NullPointerException

   [junit4]    >       at org.apache.lucene.search.TestDateSort.tearDown(TestDateSort.java:71)

   [junit4]    >       at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method)

   [junit4]    >       at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)

   [junit4]    >       at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)

   [junit4]    >       at java.base/java.lang.reflect.Method.invoke(Method.java:538)

   [junit4]    >       at java.base/java.lang.Thread.run(Thread.java:844)




Uwe Schindler

Achterdiek 19, D-28357 Bremen


eMail: uwe@thetaphi.de


From: Uwe Schindler [mailto:uwe@thetaphi.de]
Sent: Friday, December 9, 2016 4:35 PM
To: 'Rory O'Donnell' <rory.odonnell@oracle.com>
Cc: 'Dalibor Topic' <dalibor.topic@oracle.com>; balchandra.vaidya@oracle.com; abdul.kolarkunnu@oracle.com; 'Dawid Weiss' <dawid.weiss@cs.put.poznan.pl>; dev@lucene.apache.org
Subject: RE: JDK 9 b148 including a refresh of the module system is available on java.net


Hi Rory,


thanks for the message! I put build 148 in the Jenkins test rotation. Let’s see what happens.





Uwe Schindler

Achterdiek 19, D-28357 Bremen


eMail: uwe@thetaphi.de


From: Rory O'Donnell [mailto:rory.odonnell@oracle.com]
Sent: Friday, December 9, 2016 11:34 AM
rory.odonnell@oracle.com; Dalibor Topic <dalibor.topic@oracle.com>; balchandra.vaidya@oracle.com; abdul.kolarkunnu@oracle.com; Dawid Weiss <dawid.weiss@cs.put.poznan.pl>; dev@lucene.apache.org
Subject: JDK 9 b148 including a refresh of the module system is available on java.net



Hi Uwe & Dawid,

JDK 9 build b148 includes an important Refresh of the module system [1] , summary of  changes are listed here .

This refresh includes a disruptive change that is important to understand.

For those that have been trying out modules with regular JDK 9 builds then be aware that `requires public` changes to `requires transitive`. In addition, the binary representation of the module declaration (module-info.class) has changed so that you need to recompile any modules that were compiled with previous JDK 9 builds.

As things stand today in JDK 9 then you use setAccessible to break into non-public elements of any type in exported packages. However, it cannot be used to break into any type in non-exported package. The current specified behavior was a compromise for the initial integration of the module system. It is of course not very satisfactory, hence the #AwkwardStrongEncapsulation issue [2] on the JSR 376 issues list. With the updated proposal in the JSR, this refresh changes setAccessible further so that it cannot be used to break into non-public types, or non-public elements of public types, in exported packages. Code that uses setAccessible to hack into the private constructor of java.lang.invoke.MethodHandles.Lookup will be disappointed for example.

This change will expose hacks in many existing libraries and tools. As a workaround then a new command line option `--add-opens` can be used to open specific packages for "deep reflection". For example, a really popular build tool fails with this refresh because it uses setAccessible + core reflection to hack into a private field of an unmodifiable collection so that it can mutate it, facepalm! This code will continue to work as before when run with `--add-opens java.base/java.util=ALL-UNNAMED` to open the package java.util in module java.base to "all unnamed modules" (think class path).

Any help reporting issues to popular tools and libraries would be appreciated.

A debugging aid that is useful to identify issues is to run with -Dsun.reflect.debugModuleAccessChecks=true to get a stack trace when setAccessible fails, this is particularly useful when code swallows exceptions without any logging.


[1] http://mail.openjdk.java.net/pipermail/jdk9-dev/2016-November/005276.html
[2] http://openjdk.java.net/projects/jigsaw/spec/issues/#AwkwardStrongEncapsulation

Rgds,Rory O'Donnell
Quality Engineering Manager
Oracle EMEA , Dublin, Ireland