hadoop-mapreduce-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "John Gordon (JIRA)" <j...@apache.org>
Subject [jira] [Created] (MAPREDUCE-4322) Fix command-line length abort issues on Windows
Date Wed, 06 Jun 2012 21:43:22 GMT
John Gordon created MAPREDUCE-4322:

             Summary: Fix command-line length abort issues on Windows
                 Key: MAPREDUCE-4322
                 URL: https://issues.apache.org/jira/browse/MAPREDUCE-4322
             Project: Hadoop Map/Reduce
          Issue Type: Bug
          Components: tasktracker
         Environment: Windows, downstream applications with long aggregate classpaths
            Reporter: John Gordon

When a task is started on the tasktracker, it creates a small batch file to invoke java and
runs that batch.  Within the batch file, the invocation of Java currently has -classpath ${CLASSPATH}
inline to the command.  That line often exceeds 8000 characters.  This is ok for most linux
distributions because the line limit env variable is often set much higher than this.  However,
for Windows this cause cmd to abort execution.  This surfaces in Hadoop as an unknown failure
mode for the task.

I think the easiest and most natural way to fix this is to push the -classpath option into
a config file to take the longest variable part of the line and put it somewhere that scales

This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira


View raw message