trafodion-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Dave Birdsall <dave.birds...@esgyn.com>
Subject RE: avoiding a JNI call to getHbaseTableInfo?
Date Wed, 05 Aug 2015 23:28:06 GMT
Hi,

Just want to make sure I understood Eric's question. Eric, were you
considering a run-time call to get this information?

If so, as Qifan points out, this information is available to the compiler.
We just would have to get it to the run-time somehow. Usual route is through
the appropriate TDB generated by the compiler and read by the executor.

Dave

-----Original Message-----
From: Qifan Chen [mailto:qifan.chen@esgyn.com]
Sent: Wednesday, August 5, 2015 4:04 PM
To: dev <dev@trafodion.incubator.apache.org>
Subject: Re: avoiding a JNI call to getHbaseTableInfo?

Hi Eric,

You can set the CQD Hbase_index_level to a positive integer value (such as
2) to completely bypass the JNI call.  Note that the JNI call also reads in
the index level, which is a function of the Hfiles in use.

Since we cache NATable,  which is  an in-memory representation of a HBase
table, the JNI cost is only incurred once for all references to a table, per
SQL session.  If the table is redefined, we will re-read the definition and
call the JNI again.

Thanks --Qifan

On Wed, Aug 5, 2015 at 5:47 PM, Eric Owhadi <eric.owhadi@esgyn.com> wrote:

> For the small scanner feature, I first hardcoded the default
> HBaseBlock size (64KB) in the check to see if small scanner is good
> candidate to enable or not.
>
> Then struggling with the regression tests and the modifications needed
> to pass, I realized that it is better to have a feature totally
> completed to avoid the cost of dealing with regression every time you
> want to improve it.
>
> So I am trying to remove the hardcode for HBase block size, and
> realized there is a function getHbaseTableInfo that will do the job of
> getting HBase block size and index level. BUT, this function is doing
> a JNI call to the HBase layer. Look like expensive to call for every
> scan query… will likely lose the gain of small scanner if I put this call
> in the middle.
>
> I was wondering if this Hbase block size per table is not already
> cached in memory so that I can access it without fear of performance
> cost? I guess it is table meta data, and I think this is supposed to
> already be cached if I recall a previous email thread?
>
> Thanks in advance for the help,
>
> Eric
>



--
Regards, --Qifan

Mime
View raw message