mahout-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Angelo Immediata <angelo...@gmail.com>
Subject Re: Info about KMEans clustering
Date Thu, 28 Nov 2013 10:35:11 GMT
Hi Suneel
First of all thank you for your support
As I'm using maven and I chenged the HADOOP version in my POM by no more
excluding the HADOOP version related to the mahout pom (it should be the
1.1.2); now I have this dependency tree:
[INFO] --- maven-dependency-plugin:2.8:tree (default-cli) @ clusteranalysis
---
[INFO] it.eng.tz.pinf:clusteranalysis:jar:0.0.1-SNAPSHOT
[INFO] +- org.apache.mahout:mahout-core:jar:0.8:compile
[INFO] |  +- org.codehaus.jackson:jackson-core-asl:jar:1.9.12:compile
[INFO] |  +- org.codehaus.jackson:jackson-mapper-asl:jar:1.9.12:compile
[INFO] |  +- org.slf4j:slf4j-api:jar:1.7.5:compile
[INFO] |  +- org.apache.commons:commons-lang3:jar:3.1:compile
[INFO] |  +- commons-io:commons-io:jar:2.4:compile
[INFO] |  +- com.thoughtworks.xstream:xstream:jar:1.4.4:compile
[INFO] |  |  +- xmlpull:xmlpull:jar:1.1.3.1:compile
[INFO] |  |  \- xpp3:xpp3_min:jar:1.1.4c:compile
[INFO] |  +- org.apache.lucene:lucene-core:jar:4.3.0:compile
[INFO] |  +- org.apache.lucene:lucene-analyzers-common:jar:4.3.0:compile
[INFO] |  +- org.apache.mahout.commons:commons-cli:jar:2.0-mahout:compile
[INFO] |  +- org.apache.commons:commons-math3:jar:3.2:compile
[INFO] |  +- org.apache.mrunit:mrunit:jar:hadoop1:1.0.0:compile
[INFO] |  |  \- org.mockito:mockito-all:jar:1.8.5:compile
[INFO] |  \- org.apache.hadoop:hadoop-core:jar:1.1.2:compile
[INFO] |     +- commons-cli:commons-cli:jar:1.2:compile
[INFO] |     +- com.sun.jersey:jersey-core:jar:1.8:compile
[INFO] |     +- com.sun.jersey:jersey-json:jar:1.8:compile
[INFO] |     |  +- org.codehaus.jettison:jettison:jar:1.1:compile
[INFO] |     |  |  \- stax:stax-api:jar:1.0.1:compile
[INFO] |     |  +- com.sun.xml.bind:jaxb-impl:jar:2.2.3-1:compile
[INFO] |     |  |  \- javax.xml.bind:jaxb-api:jar:2.2.2:compile
[INFO] |     |  |     +- javax.xml.stream:stax-api:jar:1.0-2:compile
[INFO] |     |  |     \- javax.activation:activation:jar:1.1:compile
[INFO] |     |  +- org.codehaus.jackson:jackson-jaxrs:jar:1.7.1:compile
[INFO] |     |  \- org.codehaus.jackson:jackson-xc:jar:1.7.1:compile
[INFO] |     +- com.sun.jersey:jersey-server:jar:1.8:compile
[INFO] |     |  \- asm:asm:jar:3.1:compile
[INFO] |     +- commons-httpclient:commons-httpclient:jar:3.0.1:compile
[INFO] |     +- commons-codec:commons-codec:jar:1.4:compile
[INFO] |     +- org.apache.commons:commons-math:jar:2.1:compile
[INFO] |     +- commons-configuration:commons-configuration:jar:1.6:compile
[INFO] |     |  +- commons-collections:commons-collections:jar:3.2.1:compile
[INFO] |     |  +- commons-digester:commons-digester:jar:1.8:compile
[INFO] |     |  |  \- commons-beanutils:commons-beanutils:jar:1.7.0:compile
[INFO] |     |  \-
commons-beanutils:commons-beanutils-core:jar:1.8.0:compile
[INFO] |     +- commons-net:commons-net:jar:1.4.1:compile
[INFO] |     \- commons-el:commons-el:jar:1.0:compile
[INFO] +- org.apache.mahout:mahout-math:jar:0.8:compile
[INFO] |  \- com.google.guava:guava:jar:14.0.1:compile
[INFO] +- org.apache.mahout:mahout-collections:jar:1.0:compile
[INFO] +- org.apache.mahout:mahout-utils:jar:0.5:compile
[INFO] |  +- org.apache.solr:solr-commons-csv:jar:3.1.0:compile
[INFO] |  \- org.slf4j:slf4j-jcl:jar:1.6.0:compile
[INFO] +- org.apache.mahout:mahout-integration:jar:0.8:compile
[INFO] |  +- commons-dbcp:commons-dbcp:jar:1.4:compile
[INFO] |  +- commons-pool:commons-pool:jar:1.6:compile
[INFO] |  +- org.apache.lucene:lucene-benchmark:jar:4.3.0:compile
[INFO] |  |  +- org.apache.lucene:lucene-highlighter:jar:4.3.0:compile
[INFO] |  |  |  \- org.apache.lucene:lucene-queries:jar:4.3.0:compile
[INFO] |  |  +- org.apache.lucene:lucene-memory:jar:4.3.0:compile
[INFO] |  |  +- org.apache.lucene:lucene-queryparser:jar:4.3.0:compile
[INFO] |  |  |  \- org.apache.lucene:lucene-sandbox:jar:4.3.0:compile
[INFO] |  |  |     \- jakarta-regexp:jakarta-regexp:jar:1.4:compile
[INFO] |  |  +- org.apache.lucene:lucene-facet:jar:4.3.0:compile
[INFO] |  |  +- com.ibm.icu:icu4j:jar:49.1:compile
[INFO] |  |  +- net.sourceforge.nekohtml:nekohtml:jar:1.9.17:compile
[INFO] |  |  +- org.apache.commons:commons-compress:jar:1.4.1:compile
[INFO] |  |  \- xerces:xercesImpl:jar:2.9.1:compile
[INFO] |  +- org.mongodb:mongo-java-driver:jar:2.11.1:compile
[INFO] |  +- org.mongodb:bson:jar:2.11.1:compile
[INFO] |  +- org.apache.cassandra:cassandra-all:jar:1.2.5:compile
[INFO] |  |  +- org.xerial.snappy:snappy-java:jar:1.0.4.1:compile
[INFO] |  |  +- net.jpountz.lz4:lz4:jar:1.1.0:compile
[INFO] |  |  +- com.ning:compress-lzf:jar:0.8.4:compile
[INFO] |  |  +- commons-lang:commons-lang:jar:2.4:compile
[INFO] |  |  +-
com.googlecode.concurrentlinkedhashmap:concurrentlinkedhashmap-lru:jar:1.3:compile
[INFO] |  |  +- org.antlr:antlr:jar:3.2:compile
[INFO] |  |  |  \- org.antlr:antlr-runtime:jar:3.2:compile
[INFO] |  |  |     \- org.antlr:stringtemplate:jar:3.2:compile
[INFO] |  |  |        \- antlr:antlr:jar:2.7.7:compile
[INFO] |  |  +- org.apache.cassandra.deps:avro:jar:1.4.0-cassandra-1:compile
[INFO] |  |  |  \- org.mortbay.jetty:jetty:jar:6.1.22:compile
[INFO] |  |  |     \- org.mortbay.jetty:servlet-api:jar:2.5-20081211:compile
[INFO] |  |  +- jline:jline:jar:1.0:compile
[INFO] |  |  +- com.googlecode.json-simple:json-simple:jar:1.1:compile
[INFO] |  |  +-
com.github.stephenc.high-scale-lib:high-scale-lib:jar:1.1.2:compile
[INFO] |  |  +- org.yaml:snakeyaml:jar:1.6:compile
[INFO] |  |  +- edu.stanford.ppl:snaptree:jar:0.1:compile
[INFO] |  |  +- org.mindrot:jbcrypt:jar:0.3m:compile
[INFO] |  |  +- com.yammer.metrics:metrics-core:jar:2.0.3:compile
[INFO] |  |  +- org.apache.thrift:libthrift:jar:0.7.0:compile
[INFO] |  |  |  +- javax.servlet:servlet-api:jar:2.5:compile
[INFO] |  |  |  \- org.apache.httpcomponents:httpclient:jar:4.0.1:compile
[INFO] |  |  |     \- org.apache.httpcomponents:httpcore:jar:4.0.1:compile
[INFO] |  |  +- org.apache.cassandra:cassandra-thrift:jar:1.2.5:compile
[INFO] |  |  +- com.github.stephenc:jamm:jar:0.2.5:compile
[INFO] |  |  \- io.netty:netty:jar:3.5.9.Final:compile
[INFO] |  +- org.apache.hbase:hbase-client:jar:0.95.0:compile
[INFO] |  |  +- org.apache.hbase:hbase-common:jar:0.95.0:compile
[INFO] |  |  +- org.apache.hbase:hbase-protocol:jar:0.95.0:compile
[INFO] |  |  +- com.google.protobuf:protobuf-java:jar:2.4.1:compile
[INFO] |  |  +- org.apache.zookeeper:zookeeper:jar:3.4.5:compile
[INFO] |  |  +- org.cloudera.htrace:htrace:jar:1.50:compile
[INFO] |  |  |  \- org.mortbay.jetty:jetty-util:jar:6.1.26:compile
[INFO] |  |  \-
com.github.stephenc.findbugs:findbugs-annotations:jar:1.3.9-1:compile
[INFO] |  \- me.prettyprint:hector-core:jar:1.0-5:compile
[INFO] |     +- com.github.stephenc.eaio-uuid:uuid:jar:3.2.0:compile
[INFO] |     \- com.ecyrd.speed4j:speed4j:jar:0.9:compile
[INFO] +- org.apache.mahout:mahout-examples:jar:0.8:compile
[INFO] +- commons-logging:commons-logging:jar:1.1.3:compile
[INFO] +- log4j:log4j:jar:1.2.17:compile
[INFO] \- junit:junit:jar:4.11:test
[INFO]    \- org.hamcrest:hamcrest-core:jar:1.3:test

As we can see I have the hadoop-core version 1.1.2 in my classpath but I'm
still having that error...I can't figure what I'm wrong
The error is always:
java.lang.IllegalStateException:
file:/root/Scrivania/testKmean/input/testdata/clusters/part-00000
at
org.apache.mahout.common.iterator.sequencefile.SequenceFileDirValueIterator$1.apply(SequenceFileDirValueIterator.java:129)
at
org.apache.mahout.common.iterator.sequencefile.SequenceFileDirValueIterator$1.apply(SequenceFileDirValueIterator.java:120)
at com.google.common.collect.Iterators$8.transform(Iterators.java:860)
at
com.google.common.collect.TransformedIterator.next(TransformedIterator.java:48)
at com.google.common.collect.Iterators$5.hasNext(Iterators.java:597)
at
com.google.common.collect.ForwardingIterator.hasNext(ForwardingIterator.java:43)
at
org.apache.mahout.clustering.kmeans.KMeansUtil.configureWithClusterInfo(KMeansUtil.java:51)
at
org.apache.mahout.clustering.kmeans.KMeansDriver.buildClusters(KMeansDriver.java:209)
at
org.apache.mahout.clustering.kmeans.KMeansDriver.run(KMeansDriver.java:143)
at
it.eng.tz.pinf.clustering.ClusterAnalysisModule.executeClusterAnalysis(ClusterAnalysisModule.java:84)
at
it.eng.tz.pinf.clustering.KMeansDriverClusterTest.kmeansClusteringTest(KMeansDriverClusterTest.java:24)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at
sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
at
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:606)
at
org.junit.runners.model.FrameworkMethod$1.runReflectiveCall(FrameworkMethod.java:47)
at
org.junit.internal.runners.model.ReflectiveCallable.run(ReflectiveCallable.java:12)
at
org.junit.runners.model.FrameworkMethod.invokeExplosively(FrameworkMethod.java:44)
at
org.junit.internal.runners.statements.InvokeMethod.evaluate(InvokeMethod.java:17)
at org.junit.runners.ParentRunner.runLeaf(ParentRunner.java:271)
at
org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:70)
at
org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:50)
at org.junit.runners.ParentRunner$3.run(ParentRunner.java:238)
at org.junit.runners.ParentRunner$1.schedule(ParentRunner.java:63)
at org.junit.runners.ParentRunner.runChildren(ParentRunner.java:236)
at org.junit.runners.ParentRunner.access$000(ParentRunner.java:53)
at org.junit.runners.ParentRunner$2.evaluate(ParentRunner.java:229)
at org.junit.runners.ParentRunner.run(ParentRunner.java:309)
at
org.eclipse.jdt.internal.junit4.runner.JUnit4TestReference.run(JUnit4TestReference.java:50)
at
org.eclipse.jdt.internal.junit.runner.TestExecution.run(TestExecution.java:38)
at
org.eclipse.jdt.internal.junit.runner.RemoteTestRunner.runTests(RemoteTestRunner.java:467)
at
org.eclipse.jdt.internal.junit.runner.RemoteTestRunner.runTests(RemoteTestRunner.java:683)
at
org.eclipse.jdt.internal.junit.runner.RemoteTestRunner.run(RemoteTestRunner.java:390)
at
org.eclipse.jdt.internal.junit.runner.RemoteTestRunner.main(RemoteTestRunner.java:197)
Caused by: java.io.EOFException
at java.io.DataInputStream.readFully(DataInputStream.java:197)
at java.io.DataInputStream.readFully(DataInputStream.java:169)
at org.apache.hadoop.io.SequenceFile$Reader.init(SequenceFile.java:1512)
at org.apache.hadoop.io.SequenceFile$Reader.<init>(SequenceFile.java:1490)
at org.apache.hadoop.io.SequenceFile$Reader.<init>(SequenceFile.java:1479)
at org.apache.hadoop.io.SequenceFile$Reader.<init>(SequenceFile.java:1474)
at
org.apache.mahout.common.iterator.sequencefile.SequenceFileValueIterator.<init>(SequenceFileValueIterator.java:56)
at
org.apache.mahout.common.iterator.sequencefile.SequenceFileDirValueIterator$1.apply(SequenceFileDirValueIterator.java:124)
... 33 more


Am I missing any other passage? Should I activate HADOOP daemons? Should I
put my input and output directories in specific points?

Thank you
Angelo



2013/11/28 Suneel Marthi <suneel_marthi@yahoo.com>

> This is not an issue with Mahout and more to do with ur environment. U
> seem to be missing Hadoop in it path,
>
> Also mahout 0.8 is officially not supported on Hadoop 2.2.
>
> Sent from my iPhone
>
> > On Nov 28, 2013, at 4:39 AM, Angelo Immediata <angeloimm@gmail.com>
> wrote:
> >
> > Hi all
> > I'm pretty new to mahout and I don't know if this is the right place
> where
> > to post this questions, so pardon me if I'm wrong :)
> > I'm using apache mahout 0.8, apache hadoop 2.2.0; I wanted to test this
> > class:
> > public class ClusterAnalysisModule {
> >
> > public static final double[][] points = { { 1, 1 }, { 2, 1 }, { 1, 2 }, {
> > 2, 2 }, { 3, 3 }, { 8, 8 }, { 9, 8 }, { 8, 9 }, { 9, 9 } };
> > private static final Log LOG =
> > LogFactory.getLog(ClusterAnalysisModule.class.getName());
> > private void writePointsToFile(List<Vector> points, String fileName,
> > FileSystem fs, Configuration conf) throws IOException {
> > Path path = new Path(fileName);
> > Option fileOption = SequenceFile.Writer.file(path);
> > Option keyClassOption = SequenceFile.Writer.keyClass(LongWritable.class);
> > Option valueClassOption =
> > SequenceFile.Writer.valueClass(VectorWritable.class);
> > SequenceFile.Writer writer = SequenceFile.createWriter(conf, fileOption,
> > keyClassOption, valueClassOption);
> > long recNum = 0;
> > VectorWritable vec = new VectorWritable();
> > for (Vector point : points) {
> > vec.set(point);
> > writer.append(new LongWritable(recNum++), vec);
> > }
> > writer.close();
> > }
> >
> > private List<Vector> getPoints(double[][] raw) {
> > List<Vector> points = new ArrayList<Vector>();
> > for (int i = 0; i < raw.length; i++) {
> > double[] fr = raw[i];
> > Vector vec = new RandomAccessSparseVector(fr.length);
> > vec.assign(fr);
> > points.add(vec);
> > }
> > return points;
> > }
> >
> > public void executeClusterAnalysis() throws Exception {
> > SequenceFile.Writer writer = null;
> > SequenceFile.Reader reader = null;
> > try{
> > // Numero dei cluster in uscita
> > int k = 2;
> > List<Vector> vectors = getPoints(points); // Input data
> > Configuration conf = new Configuration();
> > FileSystem fs = FileSystem.get(conf);
> > writePointsToFile(vectors, "/root/Scrivania/testKmean/dati/file1", fs,
> > conf); // Scrivo
> > // i
> > // punti
> > // centrali
> > // iniziali
> > Path path = new
> > Path("/root/Scrivania/testKmean/input/testdata/clusters/part-00000");
> > Option fileOption = SequenceFile.Writer.file(path);
> > Option keyClassOption = SequenceFile.Writer.keyClass(Text.class);
> > Option valueClassOption = SequenceFile.Writer.valueClass(Kluster.class);
> > writer = SequenceFile.createWriter(conf, fileOption, keyClassOption,
> > valueClassOption);
> > for (int i = 0; i < k; i++) {
> > Vector vec = vectors.get(i);
> > // Cluster cluster = new Canopy(vec, i, new
> > // EuclideanDistanceMeasure());
> > Kluster cluster = new Kluster(vec, i, new EuclideanDistanceMeasure());
> > writer.append(new Text((cluster).getIdentifier()), cluster);
> > }
> > KMeansDriver.run(conf, new Path("/root/Scrivania/testKmean/dati/"),
> > new Path("/root/Scrivania/testKmean/input/testdata/clusters"),
> > new Path("/root/Scrivania/testKmean/output"), new
> > EuclideanDistanceMeasure(), 0.001, 10, true, 10, false);
> > // Path readerPath = new Path("/root/Scrivania/testKmean/output/" +
> > Cluster.CLUSTERED_POINTS_DIR + "/part-m-00000");
> > // org.apache.hadoop.io.SequenceFile.Reader.Option optReadPath =
> > org.apache.hadoop.io.SequenceFile.Reader.file(readerPath);
> > // reader = SequenceFile.createWriter(conf, optReadPath);
> > reader = new SequenceFile.Reader(fs, new
> > Path("/root/Scrivania/testKmean/output/" + Cluster.CLUSTERED_POINTS_DIR +
> > "/part-m-00000"), conf);
> > IntWritable key = new IntWritable();
> > WeightedVectorWritable value = new WeightedVectorWritable();
> > while (reader.next(key, value)) {
> > LOG.info(value.toString() + " belongs to cluster " + key.toString());
> > }
> > } catch(Exception e){
> >
> > LOG.fatal("Errore", e);
> > throw new IllegalStateException(e);
> > } finally {
> > if( writer != null ){
> >
> > writer.close();
> > }
> > if( reader != null ){
> >
> > reader.close();
> > }
> > }
> > }
> > }
> >
> > But when I execute it I get this error:
> > 10:39:14,083 DEBUG [MutableMetricsFactory] field
> > org.apache.hadoop.metrics2.lib.MutableRate
> > org.apache.hadoop.security.UserGroupInformation$UgiMetrics.loginSuccess
> > with annotation
> > @org.apache.hadoop.metrics2.annotation.Metric(valueName=Time, value=[Rate
> > of successful kerberos logins and latency (milliseconds)], about=,
> > type=DEFAULT, always=false, sampleName=Ops)
> > 10:39:14,097 DEBUG [MutableMetricsFactory] field
> > org.apache.hadoop.metrics2.lib.MutableRate
> > org.apache.hadoop.security.UserGroupInformation$UgiMetrics.loginFailure
> > with annotation
> > @org.apache.hadoop.metrics2.annotation.Metric(valueName=Time, value=[Rate
> > of failed kerberos logins and latency (milliseconds)], about=,
> > type=DEFAULT, always=false, sampleName=Ops)
> > 10:39:14,099 DEBUG [MetricsSystemImpl] UgiMetrics, User and group related
> > metrics
> > SLF4J: Class path contains multiple SLF4J bindings.
> > SLF4J: Found binding in
> >
> [jar:file:/root/.m2/repository/org/slf4j/slf4j-log4j12/1.7.5/slf4j-log4j12-1.7.5.jar!/org/slf4j/impl/StaticLoggerBinder.class]
> > SLF4J: Found binding in
> >
> [jar:file:/root/.m2/repository/org/slf4j/slf4j-jcl/1.6.0/slf4j-jcl-1.6.0.jar!/org/slf4j/impl/StaticLoggerBinder.class]
> > SLF4J: See http://www.slf4j.org/codes.html#multiple_bindings for an
> > explanation.
> > SLF4J: Actual binding is of type [org.slf4j.impl.Log4jLoggerFactory]
> > 10:39:14,355 DEBUG [KerberosName] Kerberos krb5 configuration not found,
> > setting default realm to empty
> > 10:39:14,358 DEBUG [Groups]  Creating new Groups object
> > 10:39:14,361 DEBUG [NativeCodeLoader] Trying to load the custom-built
> > native-hadoop library...
> > 10:39:14,362 DEBUG [NativeCodeLoader] Failed to load native-hadoop with
> > error: java.lang.UnsatisfiedLinkError: no hadoop in java.library.path
> > 10:39:14,362 DEBUG [NativeCodeLoader]
> >
> java.library.path=/usr/lib/jvm/jdk1.7.0_45/jre/lib/i386/server:/usr/lib/jvm/jdk1.7.0_45/jre/lib/i386:/usr/lib/jvm/jdk1.7.0_45/jre/../lib/i386:/usr/lib/jvm/jdk1.7.0_45/jre/lib/i386/client:/usr/lib/jvm/jdk1.7.0_45/jre/lib/i386::/usr/java/packages/lib/i386:/lib:/usr/lib
> > 10:39:14,362 WARN  [NativeCodeLoader] Unable to load native-hadoop
> library
> > for your platform... using builtin-java classes where applicable
> > 10:39:14,362 DEBUG [JniBasedUnixGroupsMappingWithFallback] Falling back
> to
> > shell based
> > 10:39:14,363 DEBUG [JniBasedUnixGroupsMappingWithFallback] Group mapping
> > impl=org.apache.hadoop.security.ShellBasedUnixGroupsMapping
> > 10:39:14,495 DEBUG [Groups] Group mapping
> > impl=org.apache.hadoop.security.JniBasedUnixGroupsMappingWithFallback;
> > cacheTimeout=300000
> > 10:39:14,516 DEBUG [UserGroupInformation] hadoop login
> > 10:39:14,517 DEBUG [UserGroupInformation] hadoop login commit
> > 10:39:14,521 DEBUG [UserGroupInformation] using local user:UnixPrincipal:
> > root
> > 10:39:14,527 DEBUG [UserGroupInformation] UGI loginUser:root
> (auth:SIMPLE)
> > 10:39:14,828 DEBUG [Shell] setsid exited with exit code 0
> > 10:39:14,917 INFO  [CodecPool] Got brand-new compressor [.deflate]
> > 10:39:14,945 DEBUG [CodecPool] Got recycled compressor
> > 10:39:14,959 INFO  [KMeansDriver] Input: /root/Scrivania/testKmean/dati
> > Clusters In: /root/Scrivania/testKmean/input/testdata/clusters Out:
> > /root/Scrivania/testKmean/output Distance:
> > org.apache.mahout.common.distance.EuclideanDistanceMeasure
> > 10:39:14,959 INFO  [KMeansDriver] convergence: 0.001 max Iterations: 10
> > 10:39:14,987 WARN  [FSInputChecker] Problem opening checksum file:
> > file:/root/Scrivania/testKmean/input/testdata/clusters/part-00000.
> > Ignoring exception:
> > java.io.EOFException
> > at java.io.DataInputStream.readFully(DataInputStream.java:197)
> > at java.io.DataInputStream.readFully(DataInputStream.java:169)
> > at
> >
> org.apache.hadoop.fs.ChecksumFileSystem$ChecksumFSInputChecker.<init>(ChecksumFileSystem.java:146)
> > at
> org.apache.hadoop.fs.ChecksumFileSystem.open(ChecksumFileSystem.java:339)
> > at
> org.apache.hadoop.io.SequenceFile$Reader.openFile(SequenceFile.java:1832)
> > at
> org.apache.hadoop.io.SequenceFile$Reader.<init>(SequenceFile.java:1752)
> > at
> org.apache.hadoop.io.SequenceFile$Reader.<init>(SequenceFile.java:1773)
> > at
> >
> org.apache.mahout.common.iterator.sequencefile.SequenceFileValueIterator.<init>(SequenceFileValueIterator.java:56)
> > at
> >
> org.apache.mahout.common.iterator.sequencefile.SequenceFileDirValueIterator$1.apply(SequenceFileDirValueIterator.java:124)
> > at
> >
> org.apache.mahout.common.iterator.sequencefile.SequenceFileDirValueIterator$1.apply(SequenceFileDirValueIterator.java:120)
> > at com.google.common.collect.Iterators$8.next(Iterators.java:812)
> > at com.google.common.collect.Iterators$5.hasNext(Iterators.java:544)
> > at
> >
> com.google.common.collect.ForwardingIterator.hasNext(ForwardingIterator.java:43)
> > at
> >
> org.apache.mahout.clustering.kmeans.KMeansUtil.configureWithClusterInfo(KMeansUtil.java:51)
> > at
> >
> org.apache.mahout.clustering.kmeans.KMeansDriver.buildClusters(KMeansDriver.java:209)
> > at
> >
> org.apache.mahout.clustering.kmeans.KMeansDriver.run(KMeansDriver.java:143)
> > at
> >
> it.eng.tz.pinf.clustering.ClusterAnalysisModule.executeClusterAnalysis(ClusterAnalysisModule.java:84)
> > at
> >
> it.eng.tz.pinf.clustering.KMeansDriverClusterTest.kmeansClusteringTest(KMeansDriverClusterTest.java:24)
> > at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> > at
> >
> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
> > at
> >
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
> > at java.lang.reflect.Method.invoke(Method.java:606)
> > at
> >
> org.junit.runners.model.FrameworkMethod$1.runReflectiveCall(FrameworkMethod.java:47)
> > at
> >
> org.junit.internal.runners.model.ReflectiveCallable.run(ReflectiveCallable.java:12)
> > at
> >
> org.junit.runners.model.FrameworkMethod.invokeExplosively(FrameworkMethod.java:44)
> > at
> >
> org.junit.internal.runners.statements.InvokeMethod.evaluate(InvokeMethod.java:17)
> > at org.junit.runners.ParentRunner.runLeaf(ParentRunner.java:271)
> > at
> >
> org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:70)
> > at
> >
> org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:50)
> > at org.junit.runners.ParentRunner$3.run(ParentRunner.java:238)
> > at org.junit.runners.ParentRunner$1.schedule(ParentRunner.java:63)
> > at org.junit.runners.ParentRunner.runChildren(ParentRunner.java:236)
> > at org.junit.runners.ParentRunner.access$000(ParentRunner.java:53)
> > at org.junit.runners.ParentRunner$2.evaluate(ParentRunner.java:229)
> > at org.junit.runners.ParentRunner.run(ParentRunner.java:309)
> > at
> >
> org.eclipse.jdt.internal.junit4.runner.JUnit4TestReference.run(JUnit4TestReference.java:50)
> > at
> >
> org.eclipse.jdt.internal.junit.runner.TestExecution.run(TestExecution.java:38)
> > at
> >
> org.eclipse.jdt.internal.junit.runner.RemoteTestRunner.runTests(RemoteTestRunner.java:467)
> > at
> >
> org.eclipse.jdt.internal.junit.runner.RemoteTestRunner.runTests(RemoteTestRunner.java:683)
> > at
> >
> org.eclipse.jdt.internal.junit.runner.RemoteTestRunner.run(RemoteTestRunner.java:390)
> > at
> >
> org.eclipse.jdt.internal.junit.runner.RemoteTestRunner.main(RemoteTestRunner.java:197)
> > 10:39:14,993 FATAL [ClusterAnalysisModule] Errore
> > java.lang.IllegalStateException:
> > file:/root/Scrivania/testKmean/input/testdata/clusters/part-00000
> > at
> >
> org.apache.mahout.common.iterator.sequencefile.SequenceFileDirValueIterator$1.apply(SequenceFileDirValueIterator.java:129)
> > at
> >
> org.apache.mahout.common.iterator.sequencefile.SequenceFileDirValueIterator$1.apply(SequenceFileDirValueIterator.java:120)
> > at com.google.common.collect.Iterators$8.next(Iterators.java:812)
> > at com.google.common.collect.Iterators$5.hasNext(Iterators.java:544)
> > at
> >
> com.google.common.collect.ForwardingIterator.hasNext(ForwardingIterator.java:43)
> > at
> >
> org.apache.mahout.clustering.kmeans.KMeansUtil.configureWithClusterInfo(KMeansUtil.java:51)
> > at
> >
> org.apache.mahout.clustering.kmeans.KMeansDriver.buildClusters(KMeansDriver.java:209)
> > at
> >
> org.apache.mahout.clustering.kmeans.KMeansDriver.run(KMeansDriver.java:143)
> > at
> >
> it.eng.tz.pinf.clustering.ClusterAnalysisModule.executeClusterAnalysis(ClusterAnalysisModule.java:84)
> > at
> >
> it.eng.tz.pinf.clustering.KMeansDriverClusterTest.kmeansClusteringTest(KMeansDriverClusterTest.java:24)
> > at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> > at
> >
> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
> > at
> >
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
> > at java.lang.reflect.Method.invoke(Method.java:606)
> > at
> >
> org.junit.runners.model.FrameworkMethod$1.runReflectiveCall(FrameworkMethod.java:47)
> > at
> >
> org.junit.internal.runners.model.ReflectiveCallable.run(ReflectiveCallable.java:12)
> > at
> >
> org.junit.runners.model.FrameworkMethod.invokeExplosively(FrameworkMethod.java:44)
> > at
> >
> org.junit.internal.runners.statements.InvokeMethod.evaluate(InvokeMethod.java:17)
> > at org.junit.runners.ParentRunner.runLeaf(ParentRunner.java:271)
> > at
> >
> org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:70)
> > at
> >
> org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:50)
> > at org.junit.runners.ParentRunner$3.run(ParentRunner.java:238)
> > at org.junit.runners.ParentRunner$1.schedule(ParentRunner.java:63)
> > at org.junit.runners.ParentRunner.runChildren(ParentRunner.java:236)
> > at org.junit.runners.ParentRunner.access$000(ParentRunner.java:53)
> > at org.junit.runners.ParentRunner$2.evaluate(ParentRunner.java:229)
> > at org.junit.runners.ParentRunner.run(ParentRunner.java:309)
> > at
> >
> org.eclipse.jdt.internal.junit4.runner.JUnit4TestReference.run(JUnit4TestReference.java:50)
> > at
> >
> org.eclipse.jdt.internal.junit.runner.TestExecution.run(TestExecution.java:38)
> > at
> >
> org.eclipse.jdt.internal.junit.runner.RemoteTestRunner.runTests(RemoteTestRunner.java:467)
> > at
> >
> org.eclipse.jdt.internal.junit.runner.RemoteTestRunner.runTests(RemoteTestRunner.java:683)
> > at
> >
> org.eclipse.jdt.internal.junit.runner.RemoteTestRunner.run(RemoteTestRunner.java:390)
> > at
> >
> org.eclipse.jdt.internal.junit.runner.RemoteTestRunner.main(RemoteTestRunner.java:197)
> > Caused by: java.io.EOFException
> > at java.io.DataInputStream.readFully(DataInputStream.java:197)
> > at java.io.DataInputStream.readFully(DataInputStream.java:169)
> > at org.apache.hadoop.io.SequenceFile$Reader.init(SequenceFile.java:1845)
> > at
> >
> org.apache.hadoop.io.SequenceFile$Reader.initialize(SequenceFile.java:1810)
> > at
> org.apache.hadoop.io.SequenceFile$Reader.<init>(SequenceFile.java:1759)
> > at
> org.apache.hadoop.io.SequenceFile$Reader.<init>(SequenceFile.java:1773)
> > at
> >
> org.apache.mahout.common.iterator.sequencefile.SequenceFileValueIterator.<init>(SequenceFileValueIterator.java:56)
> > at
> >
> org.apache.mahout.common.iterator.sequencefile.SequenceFileDirValueIterator$1.apply(SequenceFileDirValueIterator.java:124)
> > ... 32 more
> > 10:39:14,996 FATAL [KMeansDriverClusterTest] Errore durante il test del
> > clustering KMeans; messaggio errore: java.lang.IllegalStateException:
> > file:/root/Scrivania/testKmean/input/testdata/clusters/part-00000
> > java.lang.IllegalStateException: java.lang.IllegalStateException:
> > file:/root/Scrivania/testKmean/input/testdata/clusters/part-00000
> > at
> >
> it.eng.tz.pinf.clustering.ClusterAnalysisModule.executeClusterAnalysis(ClusterAnalysisModule.java:99)
> > at
> >
> it.eng.tz.pinf.clustering.KMeansDriverClusterTest.kmeansClusteringTest(KMeansDriverClusterTest.java:24)
> > at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> > at
> >
> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
> > at
> >
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
> > at java.lang.reflect.Method.invoke(Method.java:606)
> > at
> >
> org.junit.runners.model.FrameworkMethod$1.runReflectiveCall(FrameworkMethod.java:47)
> > at
> >
> org.junit.internal.runners.model.ReflectiveCallable.run(ReflectiveCallable.java:12)
> > at
> >
> org.junit.runners.model.FrameworkMethod.invokeExplosively(FrameworkMethod.java:44)
> > at
> >
> org.junit.internal.runners.statements.InvokeMethod.evaluate(InvokeMethod.java:17)
> > at org.junit.runners.ParentRunner.runLeaf(ParentRunner.java:271)
> > at
> >
> org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:70)
> > at
> >
> org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:50)
> > at org.junit.runners.ParentRunner$3.run(ParentRunner.java:238)
> > at org.junit.runners.ParentRunner$1.schedule(ParentRunner.java:63)
> > at org.junit.runners.ParentRunner.runChildren(ParentRunner.java:236)
> > at org.junit.runners.ParentRunner.access$000(ParentRunner.java:53)
> > at org.junit.runners.ParentRunner$2.evaluate(ParentRunner.java:229)
> > at org.junit.runners.ParentRunner.run(ParentRunner.java:309)
> > at
> >
> org.eclipse.jdt.internal.junit4.runner.JUnit4TestReference.run(JUnit4TestReference.java:50)
> > at
> >
> org.eclipse.jdt.internal.junit.runner.TestExecution.run(TestExecution.java:38)
> > at
> >
> org.eclipse.jdt.internal.junit.runner.RemoteTestRunner.runTests(RemoteTestRunner.java:467)
> > at
> >
> org.eclipse.jdt.internal.junit.runner.RemoteTestRunner.runTests(RemoteTestRunner.java:683)
> > at
> >
> org.eclipse.jdt.internal.junit.runner.RemoteTestRunner.run(RemoteTestRunner.java:390)
> > at
> >
> org.eclipse.jdt.internal.junit.runner.RemoteTestRunner.main(RemoteTestRunner.java:197)
> > Caused by: java.lang.IllegalStateException:
> > file:/root/Scrivania/testKmean/input/testdata/clusters/part-00000
> > at
> >
> org.apache.mahout.common.iterator.sequencefile.SequenceFileDirValueIterator$1.apply(SequenceFileDirValueIterator.java:129)
> > at
> >
> org.apache.mahout.common.iterator.sequencefile.SequenceFileDirValueIterator$1.apply(SequenceFileDirValueIterator.java:120)
> > at com.google.common.collect.Iterators$8.next(Iterators.java:812)
> > at com.google.common.collect.Iterators$5.hasNext(Iterators.java:544)
> > at
> >
> com.google.common.collect.ForwardingIterator.hasNext(ForwardingIterator.java:43)
> > at
> >
> org.apache.mahout.clustering.kmeans.KMeansUtil.configureWithClusterInfo(KMeansUtil.java:51)
> > at
> >
> org.apache.mahout.clustering.kmeans.KMeansDriver.buildClusters(KMeansDriver.java:209)
> > at
> >
> org.apache.mahout.clustering.kmeans.KMeansDriver.run(KMeansDriver.java:143)
> > at
> >
> it.eng.tz.pinf.clustering.ClusterAnalysisModule.executeClusterAnalysis(ClusterAnalysisModule.java:84)
> > ... 24 more
> > Caused by: java.io.EOFException
> > at java.io.DataInputStream.readFully(DataInputStream.java:197)
> > at java.io.DataInputStream.readFully(DataInputStream.java:169)
> > at org.apache.hadoop.io.SequenceFile$Reader.init(SequenceFile.java:1845)
> > at
> >
> org.apache.hadoop.io.SequenceFile$Reader.initialize(SequenceFile.java:1810)
> > at
> org.apache.hadoop.io.SequenceFile$Reader.<init>(SequenceFile.java:1759)
> > at
> org.apache.hadoop.io.SequenceFile$Reader.<init>(SequenceFile.java:1773)
> > at
> >
> org.apache.mahout.common.iterator.sequencefile.SequenceFileValueIterator.<init>(SequenceFileValueIterator.java:56)
> > at
> >
> org.apache.mahout.common.iterator.sequencefile.SequenceFileDirValueIterator$1.apply(SequenceFileDirValueIterator.java:124)
> > ... 32 more
> >
> > May anybody tell me where I'm wrong?
> >
> > Thank you
> > Angelo
>

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message