thrift-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Jake Farrell <jfarr...@apache.org>
Subject Re: Methods to improve the performance of Thrift (C#) query for Hbase
Date Thu, 16 Jan 2014 06:53:51 GMT
Hey Sanil
The filters you are talking about are specific to Apache HBase and are only
implemented using Apache Thrift as the transport. Please ask these
questions on the Apache HBase dev/user mailing list

-Jake




On Thu, Jan 16, 2014 at 1:45 AM, Sanilpnair <sanilpnair@arssoftware.com>wrote:

> Hi Jens,
>
> As we know Thrift is providing feature for filtering, this allows the user
> to perform server-side filtering when accessing HBase over Thrift.
>
> Eg: KeyOnlyFilter,FirstKeyOnlyFilter,PrefixFilter,ColumnPrefixFilter,
> MultipleColumnPrefixFilter etc
>
> Is KeyOnlyFilter/ FirstKeyOnlyFilter more faster(as compare to other thrift
> filters)?
>
>
>
> Regards
> Sanil
>
>
>
> -----Original Message-----
> From: Sanilpnair [mailto:sanilpnair@arssoftware.com]
> Sent: Thursday, January 16, 2014 11:44 AM
> To: user@thrift.apache.org
> Subject: RE: Methods to improve the performance of Thrift (C#) query for
> Hbase
>
> Hi Jens,
>
> Thank you for your quick and valuable response. I will check on Hbase side
> anyway :)
>
>
> Regards
> Sanil
>
>
> -----Original Message-----
> From: Jens Geyer [mailto:jensgeyer@hotmail.com]
> Sent: Thursday, January 16, 2014 1:42 AM
> To: user@thrift.apache.org
> Subject: Re: Methods to improve the performance of Thrift (C#) query for
> Hbase
>
> Hi again,
>
> overlooked your qeustiosn at the bottom. Sorry about that.
>
> > 1) how many ways we can improve the performance(optimization) of Thrift
> > query to hbase.
>
> Again, I suspect this part being more an HBase issue. But I may be wrong,
> have no opportunity to test it.
>
> > 2) how to cache the Thrift query response on Hbase. Can we force to cache
> > response on Hbase using thrift query(I couldn't find any additional
> > properties or attributes on Thrift scan/filter queries)
>
> Thrift does not provide any chaching mechanisms, if that is what you are
> looking for. However, you may want to serialize the data (using Thrift) and
> put the serialized data into Memcached or Redis or the like. Start with
> Redis, if you don't have any of these.
>
> > 3) Is Thrift load balancing? how to manage load balancing of Thrift
> server
> > on Master node.
>
> Same here. To some extent, load balancing this is out of scope. There are
> some mechanisms, likeTThreadedServer or TThreadPool server, providing a
> very
>
> simple, round robin load balancing across threads. If you want a load
> balancer across processes or cluster nodes, although Thrift can help you to
> achieve that, it does not provide anything like that out of the box.
>
> JensG
>
>
>
> -----Urspr√ľngliche Nachricht-----
> From: Jens Geyer
> Sent: Wednesday, January 15, 2014 9:00 PM
> To: user@thrift.apache.org
> Subject: Re: Methods to improve the performance of Thrift (C#) query for
> Hbase
>
> And you are sure that it is not an HBase problem?
> Just asking ...
>
>
> -----Urspr√ľngliche Nachricht-----
> From: Sanilpnair
> Sent: Wednesday, January 15, 2014 3:31 PM
> To: user@thrift.apache.org
> Subject: Methods to improve the performance of Thrift (C#) query for Hbase
>
> Hi All,
>
> I started Thrift coding (with c# and Hbase) recently. I need to fetch
> realtime data from the Hbase using Thrift, my problem is that "performance
> issue" with scanner - filter queries.
>
> Please let me know the performance enhancement methods while using thrift
> with c#. Sample code is given below:
>
> ---------------------------------------------------
> string TableName="testtable";
> Dictionary<byte[], byte[]> attributes = new Dictionary<byte[], byte[]>();
> TScan scanFilter = new TScan();
> scanFilter.FilterString =
> ToByteArray("((RowFilter(<=,'binary:1053_1371222000')) AND
> (RowFilter(>=,'binary:1053_1371217740'))) OR
> (RowFilter(<=,'binary:1055_1371222000')) AND
> (RowFilter(>=,'binary:1055_1371217740')))");
>
> var scanner = _client.scannerOpenWithScan(ToByteArray(TableName),
> scanFilter, attributes);
> for (var entry = _client.scannerGet(scanner); entry.Count > 0; entry =
> _client.scannerGet(scanner))
> {
>
>             foreach (var rowResult in entry)
>             {
>         string rowkey =
> Encoding.UTF8.GetString(rowResult.Row);
>                 Console.Write("{0} => ", rowkey);
> }
>
> }
> _client.scannerClose(scanner);
>
> ----------------------------------------------------
> I tried scannerGetList also, single condition on filter is faster!
>
>
> My question is that:
>
> 1) how many ways we can improve the performance(optimization) of Thrift
> query to hbase.
> 2) how to cache the Thrift query response on Hbase. Can we force to cache
> response on Hbase using thrift query(I couldn't find any additional
> properties or attributes on Thrift scan/filter queries)
> 3) Is Thrift load balancing? how to manage load balancing of Thrift server
> on Master node.
>
>
> I am looking forward to hearing from you, hope you can help me.
>
>
> Regards
> Sanil
>
>
>

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message