mahout-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Divya" <di...@k2associates.com.sg>
Subject generate similar documents in Mahout 0.4
Date Fri, 29 Oct 2010 02:02:24 GMT
Hi,

 

I need to generate similar documents .

 

Steps I am following :

Step 1

 - Convert documents into sequence files. 

Step2

 - Convert sequence files into sparse vectors.

Step3 

-Use RowSimilarityJob to get similar rows 

 

In Step 3 I am facing issues 

When I try to run it using following parameters 

Bin/mahout rowsimilarity-i
D:\MahoutResult\seq2spraseoutput\tfidf-vectors\part-r-00000  -o
D:\MahoutResult\rowsimilarityoutput -r 20  -s
SIMILARITY_TANIMOTO_COEFFICIENT  -m 10

 

I am getting following exception 

 

java.lang.IndexOutOfBoundsException: Index: 0, Size: 0

                at java.util.ArrayList.RangeCheck(ArrayList.java:547)

                at java.util.ArrayList.get(ArrayList.java:322)

                at
org.apache.hadoop.mapred.LocalJobRunner$Job.run(LocalJobRunner.java:124)

Oct 29, 2010 9:51:53 AM org.apache.hadoop.mapred.JobClient
monitorAndPrintJob

INFO:  map 0% reduce 0%

Oct 29, 2010 9:51:53 AM org.apache.hadoop.mapred.JobClient
monitorAndPrintJob

INFO: Job complete: job_local_0002

Oct 29, 2010 9:51:53 AM org.apache.hadoop.mapred.Counters log

INFO: Counters: 0

Oct 29, 2010 9:51:53 AM org.apache.hadoop.metrics.jvm.JvmMetrics init

INFO: Cannot initialize JVM Metrics with processName=JobTracker, sessionId=
- already initialized

Oct 29, 2010 9:51:54 AM org.apache.hadoop.mapred.JobClient
configureCommandLineOptions

WARNING: No job jar file set.  User classes may not be found. See
JobConf(Class) or JobConf#setJar(String).

Exception in thread "main"
org.apache.hadoop.mapreduce.lib.input.InvalidInputException: Input path does
not exist: temp/pairwiseSimilarity

                at
org.apache.hadoop.mapreduce.lib.input.FileInputFormat.listStatus(FileInputFo
rmat.java:224)

                at
org.apache.hadoop.mapreduce.lib.input.SequenceFileInputFormat.listStatus(Seq
uenceFileInputFormat.java:55)

                at
org.apache.hadoop.mapreduce.lib.input.FileInputFormat.getSplits(FileInputFor
mat.java:241)

                at
org.apache.hadoop.mapred.JobClient.writeNewSplits(JobClient.java:885)

                at
org.apache.hadoop.mapred.JobClient.submitJobInternal(JobClient.java:779)

                at org.apache.hadoop.mapreduce.Job.submit(Job.java:432)

                at
org.apache.hadoop.mapreduce.Job.waitForCompletion(Job.java:447)

                at
org.apache.mahout.math.hadoop.similarity.RowSimilarityJob.run(RowSimilarityJ
ob.java:174)

                at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:65)

                at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:79)

                at
org.apache.mahout.math.hadoop.similarity.RowSimilarityJob.main(RowSimilarity
Job.java:86)

 

 

Is there any issues with the input?

Am I following the steps correctly ?

 

Regards,

Divya 

 


Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message