giraph-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From <>
Subject RE: BUG ? SequenceFileVertexInputFormat with custom Writable : can not iterate over vertices
Date Wed, 10 Sep 2014 07:28:24 GMT
Anyone ?

-----Message d'origine-----
Envoyé : jeudi 4 septembre 2014 15:31
À :
Objet : BUG ? SequenceFileVertexInputFormat with custom Writable : can not iterate over vertices


first of all many thanks to the community for the great work you are doing.

with giraph 1.1-SNAPSHOT
built for hadoop 2.4.0

I am stalled on a bug using custom SequenceFile Vertex Input format.

I have opened a project on github to allow you to run/play/view the bug

you will find everything needed :D

I try to load/create my vertices from a SequenceFile<LongWritable,MyWritable>
"LongWritable" being the vertexId
"MyWritable" being a custom value writable used to generate VertexValue and its edges

Vertices are created correctly and loaded (as seen on the log)

2014-09-04 10:44:44,683 DEBUG [load-0] MyReader (
- Create Vertex from "1.0 0.0 0:" to : Vertex(id=1,value=1.0,#edges=0) 2014-09-04 10:44:44,694
DEBUG [load-0] MyReader ( -
Create Vertex from "2.0 0.0 2:1|6|" to : Vertex(id=2,value=2.0,#edges=2) 2014-09-04 10:44:44,695
DEBUG [load-0] MyReader ( -
Create Vertex from "5.0 0.0 3:4|1|6|" to : Vertex(id=5,value=5.0,#edges=3)

2014-09-04 10:44:44,697 INFO [load-0] worker.InputSplitsCallable (
- call: Loaded 1 input splits in 0.12198963 secs, (v=3, e=5) 24.592255 vertices/sec, 40.98709

But alas, when trying to iterate over them during a compute session, I get the following error

ERROR : But when reading in PrintVertex (BasicComputation): 2014-09-04 10:44:44,701 ERROR
[load-0] utils.LogStacktraceCallable ( - Execution of
callable failed java.lang.IllegalStateException: next: IOException at
at org.apache.giraph.partition.BasicPartition.addPartitionVertices(
at org.apache.giraph.comm.requests.SendWorkerVerticesRequest.doRequest(
at org.apache.giraph.comm.netty.NettyWorkerClientRequestProcessor.doRequest(
at org.apache.giraph.comm.netty.NettyWorkerClientRequestProcessor.flush(
at at
at at
at java.util.concurrent.ThreadPoolExecutor.runWorker( at java.util.concurrent.ThreadPoolExecutor$
at Caused by: ensureRemaining:
Only 5 bytes remaining, trying to read 8 at org.apache.giraph.utils.UnsafeByteArrayInputStream.ensureRemaining(
at org.apache.giraph.utils.UnsafeByteArrayInputStream.readLong(
at at org.apache.giraph.utils.WritableUtils.reinitializeVertexFromDataInput(
at ... 11 more

Any idea ?

Thanks a lot



Ce message et ses pieces jointes peuvent contenir des informations confidentielles ou privilegiees
et ne doivent donc
pas etre diffuses, exploites ou copies sans autorisation. Si vous avez recu ce message par
erreur, veuillez le signaler
a l'expediteur et le detruire ainsi que les pieces jointes. Les messages electroniques etant
susceptibles d'alteration,
Orange decline toute responsabilite si ce message a ete altere, deforme ou falsifie. Merci.

This message and its attachments may contain confidential or privileged information that may
be protected by law;
they should not be distributed, used or copied without authorisation.
If you have received this email in error, please notify the sender and delete this message
and its attachments.
As emails may be altered, Orange is not liable for messages that have been modified, changed
or falsified.
Thank you.

View raw message