hadoop-mapreduce-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Michael Miklavcic (JIRA)" <j...@apache.org>
Subject [jira] [Created] (MAPREDUCE-5855) Oozie jobs able to affect container classpath
Date Wed, 23 Apr 2014 21:58:14 GMT
Michael Miklavcic created MAPREDUCE-5855:
--------------------------------------------

             Summary: Oozie jobs able to affect container classpath
                 Key: MAPREDUCE-5855
                 URL: https://issues.apache.org/jira/browse/MAPREDUCE-5855
             Project: Hadoop Map/Reduce
          Issue Type: Bug
          Components: applicationmaster, job submission, mr-am, mrv2
    Affects Versions: 2.2.0
         Environment: Centos 6.4
            Reporter: Michael Miklavcic


A submitted job is able to affect the container classpath and cause failures at the container
level.

The jar spec allows a user to specify a service provider via the Service Loader interface
for things like XML implementations. These implementations can be provided by including files
in META-INF/services that have a line declaring which implementation should be loaded by the
JVM.

I've attached a contrived example that mimics what I witnessed at a client site. They had
xalan and xerces jars in an Oozie lib directory as dependencies for a Pig workflow. The stacktrace
is also attached.

There is a work-around for this problem - either include all necessary jars, e.g. serializer.jar,
in your Oozie workflow lib, or rely on default xml providers. But, is this expected behavior
from yarn/map-reduce?



--
This message was sent by Atlassian JIRA
(v6.2#6252)

Mime
View raw message