gora-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Renato MarroquĂ­n Mogrovejo <renatoj.marroq...@gmail.com>
Subject Re: Need Urgent Help with Giraph-Gora
Date Fri, 06 Oct 2017 22:16:49 GMT
Hi Adhish,

Alfonso is right, the Gora connector works on the assumption that
everything to be read has been written using Gora.
So basically it is a Gora -> DataStore -> Gora -> Giraph. But if you need
that single dummy element, Alfonso suggestion might do it.
Did you manage to get it working then?


Best,

Renato M.


2017-10-04 11:03 GMT+02:00 Alfonso Nishikawa <alfonso.nishikawa@gmail.com>:
>
> Hi,
>
> Generically speaking, you can't add data writing directly a row with "put
...". You should create a small program that creates the entity and persist
it.
> The fields at HBase are encoded and serialized (depending on the case).
>
> Your exact error is because an issue reading 'vertices:vertexId': the
long is expected to be 8 bytes when read with #fromBytes().
> When being written with Gora, it is converted to bytes with HBase's
method Byte.toBytes() [1]. As you can see, it actually writes 8 bytes.
>
> Maybe something like:
>
>     put 'graphGiraph', '0', 'vertices:vertexId', '\0\0\0\0\0\0\0\0'
>
> could work, but I assuume that you will face further problems reading
'vertices:edges'.
> My recomendation is the one from the beginning: write a small program to
write that value through Gora.
>
> [1]-
https://github.com/apache/hbase/blob/master/hbase-common/src/main/java/org/apache/hadoop/hbase/util/Bytes.java#L758
>
> Regards,
>
> Alffonso Nishikawa
>
>
> 2017-10-04 7:31 GMT-01:00 Adhish Singla <adhish.singla@research.iiit.ac.in
>:
>>
>> This is the Stack Trace to the issue mentioned in the previous reply :
>>
>> java.lang.IllegalArgumentException: offset (0) + length (8) exceed the
capacity of the array: 1
>>         at
org.apache.hadoop.hbase.util.Bytes.explainWrongLengthOrOffset(Bytes.java:543)
>>         at org.apache.hadoop.hbase.util.Bytes.toLong(Bytes.java:521)
>>         at org.apache.hadoop.hbase.util.Bytes.toLong(Bytes.java:494)
>>         at
org.apache.gora.hbase.util.HBaseByteInterface.fromBytes(HBaseByteInterface.java:88)
>>         at
org.apache.gora.hbase.store.HBaseStore.setField(HBaseStore.java:715)
>>         at
org.apache.gora.hbase.store.HBaseStore.setField(HBaseStore.java:681)
>>         at
org.apache.gora.hbase.store.HBaseStore.newInstance(HBaseStore.java:627)
>>         at
org.apache.gora.hbase.query.HBaseResult.readNext(HBaseResult.java:48)
>>         at
org.apache.gora.hbase.query.HBaseScannerResult.nextInner(HBaseScannerResult.java:54)
>>         at
org.apache.gora.query.impl.ResultBase.next(ResultBase.java:114)
>>         at
org.apache.giraph.io.gora.GoraVertexInputFormat$GoraVertexReader.nextVertex(GoraVertexInputFormat.java:227)
>>         at
org.apache.giraph.io.internal.WrappedVertexReader.nextVertex(WrappedVertexReader.java:84)
>>         at
org.apache.giraph.worker.VertexInputSplitsCallable.readInputSplit(VertexInputSplitsCallable.java:169)
>>         at
org.apache.giraph.worker.InputSplitsCallable.loadInputSplit(InputSplitsCallable.java:267)
>>         at
org.apache.giraph.worker.InputSplitsCallable.call(InputSplitsCallable.java:212)
>>         at
org.apache.giraph.worker.InputSplitsCallable.call(InputSplitsCallable.java:60)
>>         at
org.apache.giraph.utils.LogStacktraceCallable.call(LogStacktraceCallable.java:51)
>>         at java.util.concurrent.FutureTask.run(FutureTask.java:262)
>>         at
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
>>         at
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
>>         at java.lang.Thread.run(Thread.java:745)
>>
>> ----- Original Message -----
>> From: "Adhish Singla" <adhish.singla@research.iiit.ac.in>
>> To: user@gora.apache.org
>> Sent: Wednesday, October 4, 2017 12:57:21 PM
>> Subject: Re: Need Urgent Help with Giraph-Gora
>>
>> I need to have a readable dummy entry in Hbase to start my computation,
and the Gora object reader is not able to read it.
>>
>> I try to add a dummy entry to HBase using this shell command :
>>
>> "put 'graphGiraph', '0', 'vertices:vertexId', '0'"
>>
>> And when I run my code, I get this error in the Logs :
>>
>> 2017-10-04 07:16:06,210 ERROR [load-0] gora.GoraVertexInputFormat
(GoraVertexInputFormat.java:nextVertex(231)) - Error transforming vertices.
>> 2017-10-04 07:16:06,210 ERROR [load-0] gora.GoraVertexInputFormat
(GoraVertexInputFormat.java:nextVertex(232)) - offset (0) + length (8)
exceed the capacity of the array: 1
>>
>> ----- Original Message -----
>> From: "lewismc" <lewismc@apache.org>
>> To: "Adhish Singla" <adhish.singla@research.iiit.ac.in>,
user@gora.apache.org
>> Sent: Wednesday, October 4, 2017 12:48:39 PM
>> Subject: Re: Need Urgent Help with Giraph-Gora
>>
>> What exactly is thebissue you are having here?
>> From what I can tell all of the code looks good... is a log being
>> generated? Are you seeing errors? If so can you provide them?
>> Always remember you need to make sure any changes in your Avro schema are
>> compiled with the GoraCompiler.
>> Thanks
>>
>> On Tue, Oct 3, 2017 at 10:37 PM Adhish Singla <
>> adhish.singla@research.iiit.ac.in> wrote:
>>
>> > Hi,
>> >
>> > I have attached the screenshots to the Avro schema,
gora-hbase-mapping.xml
>> > and the gora-object-reader. Kindly point out to my mistake which I
probably
>> > think is in the gora-hbase-mapping.xml.
>> >
>> > Thanks and regards,
>> > Adhish
>> >
>> > ----- Original Message -----
>> > From: "lewis john mcgibbney" <lewismc@apache.org>
>> > To: "<user@gora.apache.org>" <user@gora.apache.org>
>> > Sent: Wednesday, October 4, 2017 2:56:28 AM
>> > Subject: Re: Need Urgent Help with Giraph-Gora
>> >
>> > Can you send a link to your current gora-hbase-mapping.xml and Avro
schema?
>> > If not then merely paste them below.
>> > Lewis
>> >
>> > On Tue, Oct 3, 2017 at 8:28 AM, Adhish Singla <
>> > adhish.singla@research.iiit.ac.in> wrote:
>> >
>> > > Hi,
>> > >
>> > > I was able to resolve the issue by adding the resources to the JAR
>> > itself.
>> > > Now, I am stuck at writing the gora-hbase-mapping.xml since my avro
>> > schema
>> > > uses a nested records i.e. one record stores another record. Kindly
help
>> > me
>> > > with the same.
>> > >
>> > > I would like to contribute to update giraph to use the latest
version of
>> > > gora once I am able to cover up with my coming deadline.
>> > >
>> > > Thanks for the help.
>> > > Regards,
>> > > Adhish
>> > >
>>
>> --
>> http://home.apache.org/~lewismc/
>> @hectorMcSpector
>> http://www.linkedin.com/in/lmcgibbney
>
>

Mime
View raw message