mahout-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Suneel Marthi <suneel_mar...@yahoo.com>
Subject Re: LeaseExpiredException in ABt job of SSVD
Date Wed, 05 Jun 2013 03:10:46 GMT
1. It would be helpful if u could post the actual stacktrace for this exception. 
2. Could u post the command u r using to execute ssvd? Are you working off of trunk? If not
Mahout version?
3. Are u specifying a tempPath when running ssvd? SSVD is series of jobs. It could be that
the one of the jobs is deleting the output that is being accessed by another.  This exception
could be thrown by HDFS when a job tries accessing files that don't exist anymore.

 


________________________________
 From: Ahmed Elgohary <aagohary@gmail.com>
To: user@mahout.apache.org 
Sent: Tuesday, June 4, 2013 11:00 PM
Subject: LeaseExpiredException in ABt job of SSVD
 

Hi,

I am trying to run ssvd on amazon EMR but, I am getting a
LeaseExpriedException during the execution of the ABt job. I posted about
my problem to AWS forum
(here<http://forums.aws.amazon.com/thread.jspa?threadID=126294&tstart=0>)
as I thought first that it could be a problem with EMR. Now, the reply I
got indicates that it's a problem with the ssvd implementation. I
successfully used ssvd before for decomposing many other datasets with
different parameter settings. Is it possible that for only one dataset I
get that exception?

The dataset I am used here is pubmed abstracts 8.2m x 141k. The ssvd params
I am using are: rank = 100, oversampling = 15, power iterations = 2, and
ABt blocksize = 10000. The dataset is partitioned into 36 blocks on a 10
nodes EMR cluster.

My jar just runs the code below with the arguments: /user/data/pubmed.dat
/user/pubmed/ssvd100/tmp 8200000 141043 /user/pubmed/ssvd100/out 10000 100
15 16 2

        Configuration conf = new Configuration();
        DistributedRowMatrix A = new DistributedRowMatrix(new Path(args[0]),
                new Path(args[1]), Integer.parseInt(args[2]),
                Integer.parseInt(args[3]));
        A.setConf(conf);

        SSVDSolver ssvdSolver = new SSVDSolver(conf,
                new Path[] { A.getRowPath() }, new Path(args[4]),
                Integer.parseInt(args[5]), Integer.parseInt(args[6]),
                Integer.parseInt(args[7]), Integer.parseInt(args[8]));
        ssvdSolver.setQ(Integer.parseInt(args[9]));
        ssvdSolver.setComputeV(true);
        ssvdSolver.run();



thanks,

--ahmed
Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message