hadoop-yarn-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Prabhu Joseph (Jira)" <j...@apache.org>
Subject [jira] [Created] (YARN-9965) ClassNotFoundException when auxiliary service is loaded from HDFS on subsequent restart of NM
Date Fri, 08 Nov 2019 12:14:00 GMT
Prabhu Joseph created YARN-9965:
-----------------------------------

             Summary: ClassNotFoundException when auxiliary service is loaded from HDFS on
subsequent restart of NM
                 Key: YARN-9965
                 URL: https://issues.apache.org/jira/browse/YARN-9965
             Project: Hadoop YARN
          Issue Type: Bug
            Reporter: Prabhu Joseph
            Assignee: Prabhu Joseph


Loading an auxiliary jar from a Hdfs location on a node manager works as expected on first
time. The subsequent restart fails with ClassNotFoundException
{code:java}
2019-11-08 03:59:49,256 INFO org.apache.hadoop.util.ApplicationClassLoader: classpath: []
2019-11-08 03:59:49,256 INFO org.apache.hadoop.util.ApplicationClassLoader: system classes:
[java., javax.accessibility., javax.activation., javax.activity., javax.annotation., javax.annotation.processing.,
javax.crypto., javax.imageio., javax.jws., javax.lang.model., -javax.management.j2ee., javax.management.,
javax.naming., javax.net., javax.print., javax.rmi., javax.script., -javax.security.auth.message.,
javax.security.auth., javax.security.cert., javax.security.sasl., javax.sound., javax.sql.,
javax.swing., javax.tools., javax.transaction., -javax.xml.registry., -javax.xml.rpc., javax.xml.,
org.w3c.dom., org.xml.sax., org.apache.commons.logging., org.apache.log4j., -org.apache.hadoop.hbase.,
org.apache.hadoop., core-default.xml, hdfs-default.xml, mapred-default.xml, yarn-default.xml]
2019-11-08 03:59:49,257 INFO org.apache.hadoop.service.AbstractService: Service org.apache.hadoop.yarn.server.nodemanager.containermanager.AuxServices
failed in state INITED
java.lang.ClassNotFoundException: org.apache.auxtest.AuxServiceFromHDFS
	at java.net.URLClassLoader.findClass(URLClassLoader.java:381)
	at java.lang.ClassLoader.loadClass(ClassLoader.java:424)
	at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:335)
	at java.lang.ClassLoader.loadClass(ClassLoader.java:357)
	at org.apache.hadoop.util.ApplicationClassLoader.loadClass(ApplicationClassLoader.java:189)
	at org.apache.hadoop.util.ApplicationClassLoader.loadClass(ApplicationClassLoader.java:157)
	at java.lang.Class.forName0(Native Method)
	at java.lang.Class.forName(Class.java:348)
	at org.apache.hadoop.yarn.server.nodemanager.containermanager.AuxiliaryServiceWithCustomClassLoader.getInstance(AuxiliaryServiceWithCustomClassLoader.java:169)
	at org.apache.hadoop.yarn.server.nodemanager.containermanager.AuxServices.serviceInit(AuxServices.java:270)
	at org.apache.hadoop.service.AbstractService.init(AbstractService.java:164)
	at org.apache.hadoop.service.CompositeService.serviceInit(CompositeService.java:108)
	at org.apache.hadoop.yarn.server.nodemanager.containermanager.ContainerManagerImpl.serviceInit(ContainerManagerImpl.java:321)
	at org.apache.hadoop.service.AbstractService.init(AbstractService.java:164)
	at org.apache.hadoop.service.CompositeService.serviceInit(CompositeService.java:108)
	at org.apache.hadoop.yarn.server.nodemanager.NodeManager.serviceInit(NodeManager.java:478)
	at org.apache.hadoop.service.AbstractService.init(AbstractService.java:164)
	at org.apache.hadoop.yarn.server.nodemanager.NodeManager.initAndStartNodeManager(NodeManager.java:936)
	at org.apache.hadoop.yarn.server.nodemanager.NodeManager.main(NodeManager.java:1016)
{code}
 

*AuxService.java:  Path downloaded = download.call()*

On first time - the downloaded jar file is shown as [file:/yarn/nm/nmAuxService/org.apache.auxtest.AuxServiceFromHDFS_1573205385991/aux-service-hdfs.jar|file:///yarn/nm/nmAuxService/org.apache.auxtest.AuxServiceFromHDFS_1573205385991/aux-service-hdfs.jar]

 

On Subsequent restart - it is shown as /yarn/nm/nmAuxService/org.apache.auxtest.AuxServiceFromHDFS_1573205385991/aux-service-hdfs.jar/*

Looks the previous jar local location is not removed causing the issue.



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

---------------------------------------------------------------------
To unsubscribe, e-mail: yarn-dev-unsubscribe@hadoop.apache.org
For additional commands, e-mail: yarn-dev-help@hadoop.apache.org


Mime
View raw message