hadoop-mapreduce-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Vinod Kumar Vavilapalli (JIRA)" <j...@apache.org>
Subject [jira] [Resolved] (MAPREDUCE-5716) yarn framework occurs Shell$ExitCodeException
Date Fri, 08 May 2015 22:08:01 GMT

     [ https://issues.apache.org/jira/browse/MAPREDUCE-5716?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]

Vinod Kumar Vavilapalli resolved MAPREDUCE-5716.
------------------------------------------------
    Resolution: Invalid

Exit code 127 usually means that the command is not found / the file doesn't exist. May be
your setup has issues w.r.t container-executor binary.

Let's take it to the user mailing-lists and investigate there before we declare there is a
bug. Closing this as invalid for now.

> yarn framework occurs Shell$ExitCodeException
> ---------------------------------------------
>
>                 Key: MAPREDUCE-5716
>                 URL: https://issues.apache.org/jira/browse/MAPREDUCE-5716
>             Project: Hadoop Map/Reduce
>          Issue Type: Bug
>          Components: applicationmaster
>    Affects Versions: 2.2.0
>         Environment: hadoop2.2.0,linux redhat 5.5 
>            Reporter: dimmacro
>            Assignee: dimmacro
>            Priority: Critical
>
> I use hadoop2.2.0 to run map-reduce task,at first I set the  property "mapreduce.framework.name"
with "local" in mapred-site.xml,everything goes fine ,and there is no exception,but when I
run the mapreduce task on server cluster,setting the "mapreduce.framework.name" property value
with "yarn",it shows exception belows:
>     2014-01-10 14:51:03,131 INFO ContainersLauncher #0 org.apache.hadoop.yarn.server.nodemanager.LinuxContainerExecutor:
launchContainer: [/home/hadoop/hadoop-2.2.0/bin/container-executor, hadoop, 1, application_1389336249740_0001,
container_1389336249740_000
> _01_000013, /new/hadoop/data/tmp/nm-local-dir/usercache/hadoop/appcache/application_1389336249740_0001/container_1389336249740_0001_01_000013,
/new/hadoop/data/tmp/nm-local-dir/nmPrivate/application_1389336249740_0001/container_1389336249740_0001_01_00001
> /launch_container.sh, /new/hadoop/data/tmp/nm-local-dir/nmPrivate/application_1389336249740_0001/container_1389336249740_0001_01_000013/container_1389336249740_0001_01_000013.tokens,
/new/hadoop/data/tmp/nm-local-dir/nmPrivate/container_1389336249740_0001
> 01_000013.pid, /new/hadoop/data/tmp/nm-local-dir, /home/hadoop/hadoop-2.2.0/logs/userlogs,
cgroups=none]
> 2014-01-10 14:51:03,134 INFO ContainersLauncher #2 org.apache.hadoop.yarn.server.nodemanager.LinuxContainerExecutor:
launchContainer: [/home/hadoop/hadoop-2.2.0/bin/container-executor, hadoop, 1, application_1389336249740_0001,
container_1389336249740_000
> _01_000014, /new/hadoop/data/tmp/nm-local-dir/usercache/hadoop/appcache/application_1389336249740_0001/container_1389336249740_0001_01_000014,
/new/hadoop/data/tmp/nm-local-dir/nmPrivate/application_1389336249740_0001/container_1389336249740_0001_01_00001
> /launch_container.sh, /new/hadoop/data/tmp/nm-local-dir/nmPrivate/application_1389336249740_0001/container_1389336249740_0001_01_000014/container_1389336249740_0001_01_000014.tokens,
/new/hadoop/data/tmp/nm-local-dir/nmPrivate/container_1389336249740_0001
> 01_000014.pid, /new/hadoop/data/tmp/nm-local-dir, /home/hadoop/hadoop-2.2.0/logs/userlogs,
cgroups=none]
> 2014-01-10 14:51:03,142 INFO AsyncDispatcher event handler org.apache.hadoop.yarn.server.nodemanager.containermanager.container.Container:
Container container_1389336249740_0001_01_000013 transitioned from LOCALIZED to RUNNING
> 2014-01-10 14:51:03,144 INFO AsyncDispatcher event handler org.apache.hadoop.yarn.server.nodemanager.containermanager.container.Container:
Container container_1389336249740_0001_01_000014 transitioned from LOCALIZED to RUNNING
> 2014-01-10 14:51:03,144 WARN ContainersLauncher #4 org.apache.hadoop.yarn.server.nodemanager.LinuxContainerExecutor:
Exception from container-launch with container ID: container_1389336249740_0001_01_000016
and exit code: 127
> org.apache.hadoop.util.Shell$ExitCodeException: 
> 	at org.apache.hadoop.util.Shell.runCommand(Shell.java:464)
> 	at org.apache.hadoop.util.Shell.run(Shell.java:379)
> 	at org.apache.hadoop.util.Shell$ShellCommandExecutor.execute(Shell.java:589)
> 	at org.apache.hadoop.yarn.server.nodemanager.LinuxContainerExecutor.launchContainer(LinuxContainerExecutor.java:252)
> 	at org.apache.hadoop.yarn.server.nodemanager.containermanager.launcher.ContainerLaunch.call(ContainerLaunch.java:283)
> 	at org.apache.hadoop.yarn.server.nodemanager.containermanager.launcher.ContainerLaunch.call(ContainerLaunch.java:79)
> 	at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334)
> 	at java.util.concurrent.FutureTask.run(FutureTask.java:166)
> 	at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1110)
> 	at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:603)
> 	at java.lang.Thread.run(Thread.java:636)
> 2014-01-10 14:51:03,161 WARN ContainersLauncher #2 org.apache.hadoop.yarn.server.nodemanager.LinuxContainerExecutor:
Exit code from container container_1389336249740_0001_01_000014 is : 127
> my mapred-site.xml content:
> <configuration>
> <property>
>   <name>mapreduce.jobtracker.address</name>
>   <value>local</value>
>   <description>The host and port that the MapReduce job tracker runs
>   at.  If "local", then jobs are run in-process as a single map
>   and reduce task.
>   </description>
> </property>
> <property>
>   <name>mapreduce.jobtracker.http.address</name>
>   <value>121server:50030</value>
>   <description>
>     The job tracker http server address and port the server will listen on.
>     If the port is 0 then the server will start on a free port.
>   </description>
> </property>
> <property>
>   <name>mapreduce.job.maps</name>
>   <value>40</value>
>   <description>The default number of map tasks per job.
>   Ignored when mapreduce.jobtracker.address is "local".  
>   </description>
> </property>
> <property>
>   <name>mapreduce.framework.name</name>
>   <value>classic</value>
> </property>
> </configuration>
> my yarn-site.xml content:
> <configuration>
> <!-- Site specific YARN configuration properties -->
>  <property>
>     <name>yarn.resourcemanager.resource-tracker.address</name>
>     <value>152server:8990</value>
>     <description>host is the hostname of the resource manager and 
>     port is the port on which the NodeManagers contact the Resource Manager.
>     </description>
>   </property>
>   <property>
>     <name>yarn.resourcemanager.scheduler.address</name>
>     <value>152server:8991</value>
>     <description>host is the hostname of the resourcemanager and port is the port
>     on which the Applications in the cluster talk to the Resource Manager.
>     </description>
>   </property>
>   <property>
>     <name>yarn.resourcemanager.scheduler.class</name>    <value>org.apache.hadoop.yarn.server.resourcemanager.scheduler.capacity.CapacityScheduler</value>
>     <description>In case you do not want to use the default scheduler</description>
>   </property>
>   <property>
>     <name>yarn.resourcemanager.address</name>
>     <value>152server:8993</value>
>     <description>the host is the hostname of the ResourceManager and the port is
the port on
>     which the clients can talk to the Resource Manager. </description>
>   </property>
> <property> 
> <description>The address of the RM web application.</description> 
> <name>yarn.resourcemanager.webapp.address</name> 
> <value>152server:18088</value> 
> </property> 
> <property> 
> <name>yarn.nodemanager.aux-services</name> 
> <value>mapreduce_shuffle</value> 
> </property>
> <property>
>  <name>yarn.nodemanager.aux-services.mapreduce.shuffle.class</name>
>  <value>org.apache.hadoop.mapred.ShuffleHandler</value>
> </property>
> <property>
> <name>yarn.nodemanager.resource.memory-mb</name>
> <value>5120</value>
> </property>
> </configuration>



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

Mime
View raw message