hbase-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From 崔建伟 <cuijian...@xiaomi.com>
Subject RE: Themis : implements cross-row/corss-table transaction on HBase.
Date Wed, 09 Jul 2014 13:40:35 GMT
Hi Ted:
  thanks for your concern. Good catch on ThemisScan, I have updated this comment.
  It is important to support 0.96+, we are planning to do the work. 
  It is reasonable to support append / increment as part of the transaction. As generic client
transaction API of HBase has being proposed in HBASE-11447, we will refer this jira and make
themis support more APIs.

From: Ted Yu [yuzhihong@gmail.com]
Sent: Wednesday, July 09, 2014 12:10 AM
To: dev@hbase.apache.org
Cc: user@hbase.apache.org
Subject: Re: Themis : implements cross-row/corss-table transaction on HBase.

You may want to update the comment for ThemisScan :

//a wrapper class of Put in HBase which not expose timestamp to user
public class ThemisScan extends ThemisRead {

Is there plan to support append / increment as part of the transaction ?

Currently Themis depends on 0.94.11
Is there plan to support 0.96+ releases ?


On Tue, Jul 8, 2014 at 12:34 AM, 崔建伟 <cuijianwei@xiaomi.com> wrote:

> Hi everyone, I want to introduce our open-source project Themis which
> implements cross-row/corss-table transaction on HBase.
> Themis follows google's percolator algorithm(
> http://research.google.com/pubs/pub36726.html), which provides
> ACID-compliant transaction and snapshot isolation. The cross-row
> transaction is based on HBase's single-row atomic semantics and doesn't use
> a central transaction server, so that supports linear-scalability.
> Themis depends on a timestamp server to provides global strictly
> incremental timestamp to define the order of transactions, which will be
> used to resolve the write-write and read-write conflicts. The timestamp
> server is lightweight and could achieve hight throughput(500, 000 + qps),
> and Themis will batch timestamp requests across transactions in one Rpc, so
> that it won't become the bottleneck of the system even when processing
> billions of transactions every day.
> Although Themis could be implemented totally in client-side, we adopt
> coprocessor framework of HBase to achieve higher performance. Themis
> includes a client-side library to provides transaction APIs, such as
> themisPut/themisGet/themisScan/themisDelete, and a coprocessor library
> loaded on regionserver. Therefore, Themis could be used without changing
> the code and logic of HBase.
> We have been validating the correctness of Themis for a few months by a
> AccountTransfer simulation program, which concurrently does cross-row
> transactions by transferring money among different accounts(each account is
> a row in HBase) and verifies total money of all accounts doesn't change in
> the simulation. We have also run Themis on our production environment.
> We test the performance of Themis and get comparable result as percolator.
> The single-column transaction represents the worst performance case for
> Themis compared with HBase, the result is:
> 1) For read, the performance of percolator is 90% of HBase;
> 2) For write, the performance of percolator is 23% of HBase.
> The write performance drops a lot because Themis uses two-phase commit
> protocol to achieve ACID of transaction. For multi-row write, we improve
> the performance by paralleling all writes of pre-write phase. For
> single-row write, we are optimizing two-phase commit protocol to achieve
> better performance and will update the result when it is ready. The details
> of performance result could be found in github.
> The repository and introduction of Themis include:
> 1. Themis github: https://github.com/XiaoMi/themis/. The source code,
> performance test result and user guide could be found here.
> 2. Themis jira : https://issues.apache.org/jira/browse/HBASE-10999
> 3. Chronos github: https://github.com/XiaoMi/chronos. Chronos is our
> open-source high-availability, high-performance timestamp server to provide
> global strictly incremental timestamp for Themis.
> If you find Themis interesting, please leave us comment in the mail, jira
> or github.
> Best
> cuijianwei
View raw message