nutch-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Hudson (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (NUTCH-2378) ChildFirst plugin classloader
Date Fri, 18 Aug 2017 13:43:00 GMT

    [ https://issues.apache.org/jira/browse/NUTCH-2378?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16132763#comment-16132763
] 

Hudson commented on NUTCH-2378:
-------------------------------

SUCCESS: Integrated in Jenkins build Nutch-nutchgora #1591 (See [https://builds.apache.org/job/Nutch-nutchgora/1591/])
NUTCH-2378 ChildFirst plugin classloader (contributed by Jurian (snagel: [https://github.com/apache/nutch/commit/93fb5395478e982e45e8bebbf69435db1a8ce5e7])
* (edit) src/java/org/apache/nutch/plugin/PluginClassLoader.java
* (edit) src/plugin/parse-tika/plugin.xml
NUTCH-2378 ChildFirst plugin classloader - fix jsoup-extractor: all (snagel: [https://github.com/apache/nutch/commit/e1d9191158cc2519987c5646c64eaf5a11603089])
* (delete) src/plugin/jsoup-extractor/src/test/org/apache/nutch/parse/jsoup/extractor/ViewCountNormalizer.java
* (edit) src/java/org/apache/nutch/plugin/Extension.java
* (add) src/plugin/jsoup-extractor/src/java/org/apache/nutch/parse/jsoup/extractor/ViewCountNormalizer.java
* (edit) src/plugin/jsoup-extractor/src/java/org/apache/nutch/core/jsoup/extractor/JsoupDocumentReader.java


> ChildFirst plugin classloader
> -----------------------------
>
>                 Key: NUTCH-2378
>                 URL: https://issues.apache.org/jira/browse/NUTCH-2378
>             Project: Nutch
>          Issue Type: Improvement
>          Components: plugin
>    Affects Versions: 1.13
>            Reporter: Jurian Broertjes
>            Assignee: Sebastian Nagel
>             Fix For: 2.4, 1.14
>
>         Attachments: NUTCH-2378-childfirst-plugin-classloader.patch
>
>
> While working on upgrading the indexer-elastic plugin from 2.x to 5.x, I ran into several
nasty runtime dependency issues (both local and on Hadoop). After seeking help on the mailing
list, I still was unable to resolve these issues and after digging further, decided to try
a different plugin classloader strategy. 
> The normal classloader delegates class loading requests to it's parent classloader. This
can cause all sorts of nasty runtime dependency version conflicts (jar hell, version conflicts),
since the plugin's own classloader gets queried last. The child-first classloader approach
tries to load a class from the plugin's dependencies first and when unavailable, delegates
to it's parent classloader. This fixed the issues I had.
> The new approach can give runtime LinkageErrors, but these are easily resolvable (see
the patch for a few examples)
> I've tested the new loader a bit and am curious about others' findings.



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

Mime
View raw message