hadoop-mapreduce-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Chris Nauroth (JIRA)" <j...@apache.org>
Subject [jira] [Resolved] (MAPREDUCE-5655) Remote job submit from windows to a linux hadoop cluster fails due to wrong classpath
Date Mon, 02 Dec 2013 18:35:35 GMT

     [ https://issues.apache.org/jira/browse/MAPREDUCE-5655?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel

Chris Nauroth resolved MAPREDUCE-5655.

    Resolution: Duplicate

Hi, [~padisah].  Thanks for the bug report.

I do think this is a duplicate of MAPREDUCE-4052.  The 0.23.x code line is similar to the
2.2.x code line.  It's often the case that a bug in 2.2.x is also a bug in 0.23.x.  I've just
updated MAPREDUCE-4052 to make the title clearer and indicate that it also affects version

I recommend that your participate on MAPREDUCE-4052.  There is a patch attached to that issue,
but it's a few months old, so it's likely to be out-of-date at this point.  Seeing your latest
patch would be valuable.  You can upload your patch by clicking the More button at the top
and then going through the Attach Files dialog.  The Submit Patch button is used to submit
your patch to Jenkins for a test run against current trunk.

Thanks again!

> Remote job submit from windows to a linux hadoop cluster fails due to wrong classpath
> -------------------------------------------------------------------------------------
>                 Key: MAPREDUCE-5655
>                 URL: https://issues.apache.org/jira/browse/MAPREDUCE-5655
>             Project: Hadoop Map/Reduce
>          Issue Type: Bug
>          Components: client, job submission
>    Affects Versions: 2.2.0
>         Environment: Client machine is a Windows 7 box, with Eclipse
> Remote: there is a multi node hadoop cluster, installed on Ubuntu boxes (any linux)
>            Reporter: Attila Pados
> I was trying to run a java class on my client, windows 7 developer environment, which
submits a job to the remote Hadoop cluster, initiates a mapreduce there, and then downloads
the results back to the local machine.
> General use case is to use hadoop services from a web application installed on a non-cluster
computer, or as part of a developer environment.
> The problem was, that the ApplicationMaster's startup shell script (launch_container.sh)
was generated with wrong CLASSPATH entry. Together with the java process call on the bottom
of the file, these entries were generated in windows style, using % as shell variable marker
and ; as the CLASSPATH delimiter.
> I tracked down the root cause, and found that the MrApps.java, and the YarnRunner.java
classes create these entries, and is passed forward to the ApplicationMaster, assuming that
the OS that runs these classes will match the one running the ApplicationMaster. But it's
not the case, these are in 2 different jvm, and also the OS can be different, the strings
are generated based on the client/submitter side's OS.
> I made some workaround changes to these 2 files, so i could launch my job, however there
may be more problems ahead.

This message was sent by Atlassian JIRA

View raw message