mahout-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Gokhan Capan <gkhn...@gmail.com>
Subject Re: Mahout v0.9 is not working with 2.2.0-cdh5.0.0-beta-1
Date Thu, 03 Apr 2014 18:49:43 GMT
Could you checkout the latest trunk and build the project again?

Sent from my iPhone

> On Apr 2, 2014, at 9:50, Sean Owen <srowen@gmail.com> wrote:
>
> Not sure what's going on there. Your build is somehow internally
> inconsistent at runtime since it fails on the client side due to
> mismatching Hadoop versions. I can start making up increasingly unlikely
> causes, like you are building against local copies of the artifacts that
> are old? Running clean and install should help if so.
>
> You might debug the script to print exactly what it is running.
> On Apr 2, 2014 7:59 AM, "Phan, Truong Q" <Troung.Phan@team.telstra.com>
> wrote:
>
>> The build was successful with those "warning".
>>
>> I could not run a test case from this website:
>> http://girlincomputerscience.blogspot.com.au/2010/11/apache-mahout.html
>>
>> Here are my run with the error:
>>
>>
>> +++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++
>> $ pwd
>>
>> /ora/db002/stg001/BDMSL1D/hadoop/nem-dms/devices/mahout/mahout-distribution-0.9
>>
>> $ echo $CLASSPATH
>>
>> $ echo $PATH
>>
>> /ora/db002/stg001/BDMSL1D/hadoop/nem-dms/devices/mahout/mahout-distribution-0.9/bin:/usr/local/apache-maven/apache-maven-3.2.1/bin:/usr/local/bin:/bin:/usr/bin:/home/oracle/bin:.:/usr/java/jdk1.6.0_32/bin
>>
>> $ echo $JAVA_HOME
>> /usr/java/jdk1.6.0_32
>>
>> $ env | grep HAD
>> HADOOP_CMD=/usr/bin/hadoop
>> HADOOP_HOME=/usr/lib/hadoop
>>
>> HADOOP_STREAMING=/usr/lib/hadoop-mapreduce/hadoop-streaming-2.2.0-cdh5.0.0-beta-1.jar
>>
>> $ mahout recommenditembased --input mydata.dat --usersFile user.dat
>> --numRecommendations 2 --output output/ --similarityClassname
>> SIMILARITY_PEARSON_CORRELATION
>> Running on hadoop, using /usr/lib/hadoop/bin/hadoop and HADOOP_CONF_DIR=
>> MAHOUT-JOB:
>> /ora/db002/stg001/BDMSL1D/hadoop/nem-dms/devices/mahout/mahout-distribution-0.9/examples/target/mahout-examples-0.9-job.jar
>> 14/04/02 16:52:48 INFO common.AbstractJob: Command line arguments:
>> {--booleanData=[false], --endPhase=[2147483647], --input=[mydata.dat],
>> --maxPrefsInItemSimilarity=[500], --maxPrefsPerUser=[10],
>> --maxSimilaritiesPerItem=[100], --minPrefsPerUser=[1],
>> --numRecommendations=[2], --output=[output/],
>> --similarityClassname=[SIMILARITY_PEARSON_CORRELATION], --startPhase=[0],
>> --tempDir=[temp], --usersFile=[user.dat]}
>> 14/04/02 16:52:48 INFO common.AbstractJob: Command line arguments:
>> {--booleanData=[false], --endPhase=[2147483647], --input=[mydata.dat],
>> --minPrefsPerUser=[1], --output=[temp/preparePreferenceMatrix],
>> --ratingShift=[0.0], --startPhase=[0], --tempDir=[temp]}
>> 14/04/02 16:52:49 INFO Configuration.deprecation: mapred.input.dir is
>> deprecated. Instead, use mapreduce.input.fileinputformat.inputdir
>> 14/04/02 16:52:49 INFO Configuration.deprecation:
>> mapred.compress.map.output is deprecated. Instead, use
>> mapreduce.map.output.compress
>> 14/04/02 16:52:49 INFO Configuration.deprecation: mapred.output.dir is
>> deprecated. Instead, use mapreduce.output.fileoutputformat.outputdir
>> Exception in thread "main" java.lang.IncompatibleClassChangeError: Found
>> interface org.apache.hadoop.mapreduce.JobContext, but class was expected
>>        at
>> org.apache.mahout.common.HadoopUtil.getCustomJobName(HadoopUtil.java:174)
>>        at
>> org.apache.mahout.common.AbstractJob.prepareJob(AbstractJob.java:614)
>>        at
>> org.apache.mahout.cf.taste.hadoop.preparation.PreparePreferenceMatrixJob.run(PreparePreferenceMatrixJob.java:73)
>>        at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:70)
>>        at
>> org.apache.mahout.cf.taste.hadoop.item.RecommenderJob.run(RecommenderJob.java:164)
>>        at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:70)
>>        at
>> org.apache.mahout.cf.taste.hadoop.item.RecommenderJob.main(RecommenderJob.java:322)
>>        at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
>>        at
>> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
>>        at
>> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
>>        at java.lang.reflect.Method.invoke(Method.java:597)
>>        at
>> org.apache.hadoop.util.ProgramDriver$ProgramDescription.invoke(ProgramDriver.java:72)
>>        at org.apache.hadoop.util.ProgramDriver.run(ProgramDriver.java:144)
>>        at
>> org.apache.hadoop.util.ProgramDriver.driver(ProgramDriver.java:152)
>>        at
>> org.apache.mahout.driver.MahoutDriver.main(MahoutDriver.java:195)
>>        at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
>>        at
>> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
>>        at
>> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
>>        at java.lang.reflect.Method.invoke(Method.java:597)
>>        at org.apache.hadoop.util.RunJar.main(RunJar.java:212)
>> oracle@bpdevdmsdbs01: ----
>> /ora/db002/stg001/BDMSL1D/hadoop/nem-dms/devices/mahout/mahout-distribution-0.9
>> -----
>>
>>
>> Thanks and Regards,
>> Truong Phan
>>
>>
>> P    + 61 2 8576 5771
>> M   + 61 4 1463 7424
>> E    troung.phan@team.telstra.com
>> W  www.telstra.com
>>
>>
>>
>> -----Original Message-----
>> From: Sean Owen [mailto:srowen@gmail.com]
>> Sent: Wednesday, 2 April 2014 4:05 PM
>> To: Mahout User List
>> Subject: Re: Mahout v0.9 is not working with 2.2.0-cdh5.0.0-beta-1
>>
>> Hm, OK something sounds wrong with your directory structure, given the
>> warnings. I assumed this was changed. It could be that the .tar.gz
>> distribution isn't quite correctly set up for building from source.
>>
>> The compilation here is nothing to do with Hadoop. You show a successful
>> build; what's the part that fails?
>>
>> On Wed, Apr 2, 2014 at 6:59 AM, Phan, Truong Q <
>> Troung.Phan@team.telstra.com> wrote:
>>> Where did I modifying the build?
>>> Here are my steps of the build.
>>> I got the source from one of the official mirror website and build it.
>>> The only one exception here is that I am using the Cloudera CDH 5.0.
>>> This latest CDHv5.0 might not work with Mahout v0.9.
>>>
>>> ++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++
>>> Install Mahout
>>>
>>> $  javac -version
>>> javac 1.6.0_32
>>>
>>> $ cd /ora/db002/stg001/BDMSL1D/hadoop/nem-dms/devices/mahout
>>>
>>> $ wget
>>> http://mirror.mel.bkb.net.au/pub/apache/mahout/0.9/mahout-distribution
>>> -0.9-src.tar.gz
>>>
>>> $ mv mahout-distribution-0.9 mahout-distribution-0.9.old
>>>
>>> $ tar xvf mahout-distribution-0.9-src.tar.gz
>>>
>>> $ cd mahout-distribution-0.9
>>>
>>> $ ls
>>> bin  buildtools  core  distribution  examples  integration
>>> LICENSE.txt  math  math-scala  NOTICE.txt  pom.xml  README.txt  src
>>>
>>> $ mvn clean install -Dhadoop2.version=2.2.0-cdh5.0.0-beta-1
>> -DskipTests=true [INFO] Scanning for projects...
>>> ............
>>>
>>> [INFO]
>>> ----------------------------------------------------------------------
>>> --
>>> [INFO] Reactor Summary:
>>> [INFO]
>>> [INFO] Mahout Build Tools ................................ SUCCESS [
>>> 7.235 s] [INFO] Apache Mahout .....................................
>>> SUCCESS [  1.017 s] [INFO] Mahout Math
>>> ....................................... SUCCESS [15:46 min] [INFO]
>>> Mahout Core ....................................... SUCCESS [24:29
>>> min] [INFO] Mahout Integration ................................
>>> SUCCESS [03:38 min] [INFO] Mahout Examples
>>> ................................... SUCCESS [02:40 min] [INFO] Mahout
>>> Release Package ............................ SUCCESS [  0.075 s]
>>> [INFO] Mahout Math/Scala wrappers ........................ SUCCESS
>>> [01:12 min] [INFO]
>>> ----------------------------------------------------------------------
>>> --
>>> [INFO] BUILD SUCCESS
>>> [INFO]
>>> ----------------------------------------------------------------------
>>> --
>>> [INFO] Total time: 47:57 min
>>> [INFO] Finished at: 2014-04-02T15:07:50+10:00 [INFO] Final Memory:
>>> 49M/288M [INFO]
>>> ----------------------------------------------------------------------
>>> --
>>>
>>>
>>> Thanks and Regards,
>>> Truong Phan
>>>
>>>
>>> P    + 61 2 8576 5771
>>> M   + 61 4 1463 7424
>>> E    troung.phan@team.telstra.com
>>> W  www.telstra.com
>>>
>>>
>>>
>>> -----Original Message-----
>>> From: Sean Owen [mailto:srowen@gmail.com]
>>> Sent: Wednesday, 2 April 2014 3:33 PM
>>> To: Mahout User List
>>> Subject: Re: Mahout v0.9 is not working with 2.2.0-cdh5.0.0-beta-1
>>>
>>> This may be getting to you're-on-your-own-territory since you're
>> modifying the build. This error means your directory structure doesn't
>> match up with declarations. You said somewhere that the parent of module X
>> was Y, but the location given points to the pom of a module that isn't Y.
>>>
>>> On Wed, Apr 2, 2014 at 5:28 AM, Phan, Truong Q <
>> Troung.Phan@team.telstra.com> wrote:
>>>> Hi Sean,
>>>>
>>>> I am trying to build the Mahout again and got some WARNINGs so far.
>>>> Can you give me some hints what I have done wrong here?
>>>>
>>>> Thanks for your help so far.
>>

Mime
View raw message