hadoop-mapreduce-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Praveen Sripati <praveensrip...@gmail.com>
Subject Re: Not able to execute MRv2
Date Tue, 09 Aug 2011 14:57:53 GMT

For some reason none of the .log files are getting generated, so I had to
put some println and figure out the problems. I just see the .out files for
the RM, NM and the HistoryServer.

1)  I get the below error while starting the ResourceManager. In the
hadoop-mapreduce-1.0-SNAPSHOT/lib folder, there were avro-1.3.2.jar and
avro-1.4.1.jar files. Removing the 1.3.2 version of avro fixed the problem.
When I run JPS, I see RM, NM and HistoryServer.

java.lang.NoSuchMethodError: org.apache.avro.ipc.Server.start()V

log4j:WARN No appenders could be found for logger
log4j:WARN Please initialize the log4j system properly.

2) To run the sample program, had to create symbolic links for some of the
jars in the

When I ran the "hadoop classpath" command, it had some of the jars in the
above folder.

What would be the reason for the log files not being generated? I would like
to see the .log files to see the flow of the MRv2.


On Tue, Aug 9, 2011 at 8:57 AM, Praveen Sripati <praveensripati@gmail.com>wrote:

> Hi,
> Looks like the below instructions are a bit outdated. I got the mapreduce
> code from the MR-279 branch and the rest of the code from trunk. The
> hadoop-mapreduce-1.0-SNAPSHOT-all.tar.gz file got generated successfully.
> http://svn.apache.org/repos/asf/hadoop/common/branches/MR-279/mapreduce/INSTALL
> 1) Expanded the tar file to the $YARN_INSTALL directory, created symbolic
> links to the jars in the hadoop-common/lib folder and ran the
> "bin/yarn-daemon.sh start resourcemanager" command from the $YARN_INSTALL
> directory to get the below error
> Exception in thread "main" java.lang.NoClassDefFoundError:
> org/apache/hadoop/conf/Configuration
> Caused by: java.lang.ClassNotFoundException:
> org.apache.hadoop.conf.Configuration
>     at java.net.URLClassLoader$1.run(URLClassLoader.java:202)
>     at java.security.AccessController.doPrivileged(Native Method)
>     at java.net.URLClassLoader.findClass(URLClassLoader.java:190)
>     at java.lang.ClassLoader.loadClass(ClassLoader.java:306)
>     at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:301)
>     at java.lang.ClassLoader.loadClass(ClassLoader.java:247)
> Could not find the main class:
> org.apache.hadoop.yarn.server.resourcemanager.ResourceManager.  Program will
> exit.
> Then copied the hadoop-common-0.23.0-SNAPSHOT.jar from the
> hadoop-common/target to the hadoop-common/lib folder. Then the
> yarn-praveensripati-resourcemanager-praveensripati.out had the following
> error after starting the ResourceManager.
> Exception in thread "main" java.lang.IllegalStateException: For this
> operation, current State must be STARTED instead of INITED
>         at
> org.apache.hadoop.yarn.service.AbstractService.ensureCurrentState(AbstractService.java:101)
>         at
> org.apache.hadoop.yarn.service.AbstractService.stop(AbstractService.java:69)
>         at
> org.apache.hadoop.yarn.server.resourcemanager.amlauncher.ApplicationMasterLauncher.stop(ApplicationMasterLauncher.java:90)
>         at
> org.apache.hadoop.yarn.service.CompositeService.stop(CompositeService.java:89)
>         at
> org.apache.hadoop.yarn.server.resourcemanager.ResourceManager.stop(ResourceManager.java:423)
>         at
> org.apache.hadoop.yarn.server.resourcemanager.ResourceManager.main(ResourceManager.java:493)
> Started the NodeManager and the HistoryServer. Running the jps shows only
> the ResourceManager and the NodeManager and not the HistoryServer.
> 2) From the hadoop-common/target/hadoop-common-0.23.0-SNAPSHOT-bin/bin
> folder I get the following error running the sample program
> ./hadoop jar
> $HADOOP_MAPRED_HOME/build/hadoop-mapred-examples-0.23.0-SNAPSHOT.jar
> randomwriter -Dmapreduce.job.user.name=$USER
> -Dmapreduce.randomwriter.bytespermap=10000 -Ddfs.blocksize=536870912
> -Ddfs.block.size=536870912 -libjars
> $YARN_INSTALL/modules/hadoop-mapreduce-client-jobclient-1.0-SNAPSHOT.jar
> output
> java.lang.NoClassDefFoundError:
> org/apache/hadoop/mapreduce/lib/partition/InputSampler$Sampler
>     at java.lang.Class.getDeclaredMethods0(Native Method)
>     at java.lang.Class.privateGetDeclaredMethods(Class.java:2427)
>     at java.lang.Class.getMethod0(Class.java:2670)
>     at java.lang.Class.getMethod(Class.java:1603)
>     at
> org.apache.hadoop.util.ProgramDriver$ProgramDescription.<init>(ProgramDriver.java:60)
> The hadoop-mapreduce-client-core-1.0-SNAPSHOT.jar which has the
> InputSampler.java class is there in the hadoop-common/lib folder, but still
> getting the above error.
> None of the .log files are getting generated. So, it's getting difficult yo
> debug. Appreciate help in resolving this.
> Thanks,
> Praveen

  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message