gora-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Renato Marroquín Mogrovejo <renatoj.marroq...@gmail.com>
Subject Re: Progress percentage for gora-hbase record readers
Date Wed, 07 May 2014 16:51:31 GMT
Hi Alparslan,

I think this is because we are not able to know before hand how many tuples
are going to be read or fetched. We could do this but then we would have to
do an extra operation to figure this out.
I think this is the same behaviour inside the MapReduce examples from HBase
itself [1]. Maybe if we knew the maximum number of records that a request
is going to get, then we would be able to tell how many we have already
seen, and then pass this result as the request progress.


Renato M.

[1]
https://github.com/apache/hbase/blob/trunk/hbase-server/src/main/java/org/apache/hadoop/hbase/mapreduce/TableRecordReaderImpl.java#L297


2014-05-03 10:02 GMT+02:00 Alparslan Avcı <alparslanavci@gmail.com>:

> Hi all,
>
> While upgrading Nutch 2.x to gora-0.4, we have noticed that gora-hbase
> always returns 0 (zero) for progress percentage when GoraRecordReader is
> used in Mappers. And when I looked into the HBaseScannerResult, which
> handles and calculates the progress of the result of a query based on an
> HBase scanner, I've seen that there is a TODO in getProgress() method.
> Moreoever, it has not been updated from the beginning of the project.
>
> Is there any work-on-progress or discussion about this issue?
>
> Thanks.
>
> --
> Alparslan Avcı
>

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message