mahout-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Rajpal, Harjeet Kumar" <Harjeet.Ku...@Honeywell.com>
Subject RE: Mahout on Hadoop
Date Tue, 08 Dec 2009 07:05:17 GMT
I just read the code in Recommender and saw that userIdfile path is
given as input to mapred.input.dir. if I only give --userIdFile /input/
instead of --userIdFile /input/users.dat. then it still gives number
format error.

09/12/08 12:27:10 INFO mapred.JobClient: Running job:
job_200912072136_0008
09/12/08 12:27:11 INFO mapred.JobClient:  map 0% reduce 0%
09/12/08 12:29:02 INFO mapred.JobClient: Task Id :
attempt_200912072136_0008_m_000000_0, Status : FAILED
java.lang.NumberFormatException: For input string: "23,3920,2"
	at
java.lang.NumberFormatException.forInputString(NumberFormatException.jav
a:48)
	at java.lang.Long.parseLong(Long.java:412)
	at java.lang.Long.parseLong(Long.java:461)
	at
org.apache.mahout.cf.taste.hadoop.RecommenderMapper.map(RecommenderMappe
r.java:104)
	at
org.apache.mahout.cf.taste.hadoop.RecommenderMapper.map(RecommenderMappe
r.java:52)
	at org.apache.hadoop.mapred.MapRunner.run(MapRunner.java:50)
	at
org.apache.hadoop.mapred.MapTask.runOldMapper(MapTask.java:358)
	at org.apache.hadoop.mapred.MapTask.run(MapTask.java:307)
	at org.apache.hadoop.mapred.Child.main(Child.java:170)

09/12/08 12:29:02 INFO mapred.JobClient: Task Id :
attempt_200912072136_0008_m_000001_0, Status : FAILED
java.lang.NumberFormatException: For input string: "1,1193,5"
	at
java.lang.NumberFormatException.forInputString(NumberFormatException.jav
a:48)
	at java.lang.Long.parseLong(Long.java:412)
	at java.lang.Long.parseLong(Long.java:461)
	at
org.apache.mahout.cf.taste.hadoop.RecommenderMapper.map(RecommenderMappe
r.java:104)
	at
org.apache.mahout.cf.taste.hadoop.RecommenderMapper.map(RecommenderMappe
r.java:52)
	at org.apache.hadoop.mapred.MapRunner.run(MapRunner.java:50)
	at
org.apache.hadoop.mapred.MapTask.runOldMapper(MapTask.java:358)
	at org.apache.hadoop.mapred.MapTask.run(MapTask.java:307)
	at org.apache.hadoop.mapred.Child.main(Child.java:170)

09/12/08 12:29:14 INFO mapred.JobClient: Task Id :
attempt_200912072136_0008_m_000001_1, Status : FAILED
java.lang.NumberFormatException: For input string: "1,1193,5"
	at
java.lang.NumberFormatException.forInputString(NumberFormatException.jav
a:48)
	at java.lang.Long.parseLong(Long.java:412)
	at java.lang.Long.parseLong(Long.java:461)
	at
org.apache.mahout.cf.taste.hadoop.RecommenderMapper.map(RecommenderMappe
r.java:104)
	at
org.apache.mahout.cf.taste.hadoop.RecommenderMapper.map(RecommenderMappe
r.java:52)
	at org.apache.hadoop.mapred.MapRunner.run(MapRunner.java:50)
	at
org.apache.hadoop.mapred.MapTask.runOldMapper(MapTask.java:358)
	at org.apache.hadoop.mapred.MapTask.run(MapTask.java:307)
	at org.apache.hadoop.mapred.Child.main(Child.java:170)

09/12/08 12:29:20 INFO mapred.JobClient: Task Id :
attempt_200912072136_0008_m_000000_1, Status : FAILED
java.lang.NumberFormatException: For input string: "23,3920,2"
	at
java.lang.NumberFormatException.forInputString(NumberFormatException.jav
a:48)
	at java.lang.Long.parseLong(Long.java:412)
	at java.lang.Long.parseLong(Long.java:461)
	at
org.apache.mahout.cf.taste.hadoop.RecommenderMapper.map(RecommenderMappe
r.java:104)
	at
org.apache.mahout.cf.taste.hadoop.RecommenderMapper.map(RecommenderMappe
r.java:52)
	at org.apache.hadoop.mapred.MapRunner.run(MapRunner.java:50)
	at
org.apache.hadoop.mapred.MapTask.runOldMapper(MapTask.java:358)
	at org.apache.hadoop.mapred.MapTask.run(MapTask.java:307)
	at org.apache.hadoop.mapred.Child.main(Child.java:170)

09/12/08 12:29:26 INFO mapred.JobClient: Task Id :
attempt_200912072136_0008_m_000000_2, Status : FAILED
java.lang.NumberFormatException: For input string: "23,3920,2"
	at
java.lang.NumberFormatException.forInputString(NumberFormatException.jav
a:48)
	at java.lang.Long.parseLong(Long.java:412)
	at java.lang.Long.parseLong(Long.java:461)
	at
org.apache.mahout.cf.taste.hadoop.RecommenderMapper.map(RecommenderMappe
r.java:104)
	at
org.apache.mahout.cf.taste.hadoop.RecommenderMapper.map(RecommenderMappe
r.java:52)
	at org.apache.hadoop.mapred.MapRunner.run(MapRunner.java:50)
	at
org.apache.hadoop.mapred.MapTask.runOldMapper(MapTask.java:358)
	at org.apache.hadoop.mapred.MapTask.run(MapTask.java:307)
	at org.apache.hadoop.mapred.Child.main(Child.java:170)

09/12/08 12:29:26 INFO mapred.JobClient: Task Id :
attempt_200912072136_0008_m_000001_2, Status : FAILED
java.lang.NumberFormatException: For input string: "1,1193,5"
	at
java.lang.NumberFormatException.forInputString(NumberFormatException.jav
a:48)
	at java.lang.Long.parseLong(Long.java:412)
	at java.lang.Long.parseLong(Long.java:461)
	at
org.apache.mahout.cf.taste.hadoop.RecommenderMapper.map(RecommenderMappe
r.java:104)
	at
org.apache.mahout.cf.taste.hadoop.RecommenderMapper.map(RecommenderMappe
r.java:52)
	at org.apache.hadoop.mapred.MapRunner.run(MapRunner.java:50)
	at
org.apache.hadoop.mapred.MapTask.runOldMapper(MapTask.java:358)
	at org.apache.hadoop.mapred.MapTask.run(MapTask.java:307)
	at org.apache.hadoop.mapred.Child.main(Child.java:170)

09/12/08 12:29:41 INFO mapred.JobClient: Job complete:
job_200912072136_0008
09/12/08 12:29:41 INFO mapred.JobClient: Counters: 3
09/12/08 12:29:41 INFO mapred.JobClient:   Job Counters 
09/12/08 12:29:41 INFO mapred.JobClient:     Launched map tasks=8
09/12/08 12:29:41 INFO mapred.JobClient:     Data-local map tasks=8
09/12/08 12:29:41 INFO mapred.JobClient:     Failed map tasks=1
Exception in thread "main" java.io.IOException: Job failed!
	at
org.apache.hadoop.mapred.JobClient.runJob(JobClient.java:1252)
	at
org.apache.mahout.cf.taste.hadoop.RecommenderJob.main(RecommenderJob.jav
a:178)

Honestly, This is for me is confusing. I tried to figure out solution
but could not.:( 

-----Original Message-----
From: Rajpal, Harjeet Kumar [mailto:Harjeet.Kumar@Honeywell.com] 
Sent: Tuesday, December 08, 2009 12:02 PM
To: mahout-user@lucene.apache.org
Subject: RE: Mahout on Hadoop

I mean if I am right both ratings and userid file are needed as input to
recommenderjob.as per cfollowing command.

Bin/hadoop jar mahout-examples-0.3-SNAPSHOT.job
org.apache.mahout.cf.taste.hadoop.RecommenderJob  --recommenderClassName
org.apache.mahout.cf.taste.example.grouplens.GroupLensRecommender
--userRec 5 --userIdFile /input/users.dat --dataModelFile
/input/ratings.dat --output reducerOutputDir --jarFile
mahout-examples-0.3-SNAPSHOT.job

The command in previous message was wrong. Sorry for that.

-----Original Message-----
From: Rajpal, Harjeet Kumar [mailto:Harjeet.Kumar@Honeywell.com] 
Sent: Tuesday, December 08, 2009 11:55 AM
To: mahout-user@lucene.apache.org
Subject: RE: Mahout on Hadoop

This is format of users.dat file from group lens data.

"1,F,1,10,48067" . format of rating file is right as userID,itemID,pref.


Bin/hadoop jar mahout-examples-0.3-SNAPSHOT.job  --recommenderClassName
org.apache.mahout.cf.taste.example.grouplens.GroupLensRecommender
--userRec 5 --userIdFile /input/users.dat --dataModelFile
/input/ratings.dat --output reducerOutputDir --jarFile
mahout-examples-0.3-SNAPSHOT.job 

Best Regards,
Harjeet Kumar


-----Original Message-----
From: Sean Owen [mailto:srowen@gmail.com] 
Sent: Monday, December 07, 2009 9:55 PM
To: mahout-user@lucene.apache.org
Subject: Re: Mahout on Hadoop

Yes, use the GroupLens 100K data set.

I don't know what data file you are passing, but this does not appear
to be a valid preference file, what is this? Should be
userID,itemID,pref. You may need to transform the input.


On Mon, Dec 7, 2009 at 4:14 PM, Rajpal, Harjeet Kumar
<Harjeet.Kumar@honeywell.com> wrote:
>
> Ya you are right. I corrected that mistake. Sorry for inconvenience.
> I tried this with Grouplens data. It gave java.lang.OutOfMemoryError.
> can you suggest any smaller amount of data for this. I tried to use a
> part of grouplens data. Then it gave number format exception.
> "1,F,1,10,48067" .
>
> If you can provide some demo data. That will be helpful.
> Best Regards,
> Harjeet Kumar
>
> -----Original Message-----
> From: Sean Owen [mailto:srowen@gmail.com]
> Sent: Monday, December 07, 2009 3:17 PM
> To: mahout-user@lucene.apache.org
> Subject: Re: Mahout on Hadoop
>
> Yes, that's right, you want to use the .job file as your ".jar" file.
> Are you passing it both as the argument to "hadoop jar" and the
> "--jarFile" argument? I suspect you still have some deployment problem
> since this all works for me.
>
> On an unrelated note, I am changing this part of the code dramatically
> right now, improving it and cleaning it up. You may want to update at
> some point but it will likely break what you are doing a little bit.
>
> On Mon, Dec 7, 2009 at 9:18 AM, Rajpal, Harjeet Kumar
> <Harjeet.Kumar@honeywell.com> wrote:
>> I have extracted the mahout-core-snapshot.job and checked the lib.
The
>> relevant uncommons-maths-1.2.jar is inside that.
>> I hope I am not making any stupid mistake here. If I am, please
> correct
>> me. Best Regards,
>> Harjeet Kumar Rajpal
>>
>> -----Original Message-----
>> From: Sean Owen [mailto:srowen@gmail.com]
>> Sent: Monday, December 07, 2009 1:02 PM
>> To: mahout-user@lucene.apache.org
>> Subject: Re: Mahout on Hadoop
>>
>> Caused by: java.lang.ClassNotFoundException:
>> org.uncommons.maths.random.MersenneTwisterRNG
>>
>> You aren't bundling the dependent class files with the .jar file used
>> with the Hadoop job.
>>
>> On Mon, Dec 7, 2009 at 7:01 AM, Rajpal, Harjeet Kumar
>> <Harjeet.Kumar@honeywell.com> wrote:
>>> Hi Sean,
>>>
>>> Sorry for Delay in Reply.
>>> I got the new code now. The error has changed.
>>>
>>
>

Mime
View raw message