trafodion-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Dave Birdsall <dave.birds...@esgyn.com>
Subject RE: Secondary index maintenance/management code in Trafodion..
Date Wed, 28 Jun 2017 21:50:27 GMT
Hi,

The transaction is initiated by the client, but transaction processing happens at various
places in the stack.

A "tm" process on the node where the client is co-ordinates the transaction, managing the
details of the two-phase commit protocol.

Each HBase RegionServer functions as a resource manager within two-phase commit.

A Trafodion co-processor runs in each RegionServer managing the details of a given transaction
for affected rows stored at that RegionServer. Changed rows are kept in memory until transaction
commit.

Trafodion uses the MVCC model for transactions. Rows are not changed in the HBase tables until
transaction commit time.

Dave

-----Original Message-----
From: Devaraj Das [mailto:ddas@hortonworks.com] 
Sent: Wednesday, June 28, 2017 2:45 PM
To: dev@trafodion.incubator.apache.org
Subject: Re: Secondary index maintenance/management code in Trafodion..

Hi Dave, thanks for the nice details. Just to be sure I get it right, the data/index transactions
for INSERTs/UPDATEs gets executed on the client side, right?
________________________________________
From: Dave Birdsall <dave.birdsall@esgyn.com>
Sent: Wednesday, June 28, 2017 2:34 PM
To: dev@trafodion.incubator.apache.org
Subject: RE: Secondary index maintenance/management code in Trafodion..

Hi,

Secondary indexes are managed by the Trafodion engine itself; HBase is not aware of them.

>From an HBase perspective, a Trafodion secondary index is simply another HBase table.

The Trafodion engine provides a suite of DDL for the management of secondary indexes. One
can create them (via CREATE INDEX), drop them (via DROP INDEX) and alter certain attributes
(via ALTER INDEX). These DDL operations are documented in the Trafodion SQL Reference manual,
here: http://trafodion.apache.org/docs/sql_reference/index.html

>From a DML perspective, the Trafodion compiler and optimizer decide when to use a particular
secondary index. An INSERT operation of course will do an insert into all secondary indexes
as well as the base table. Similarly for a DELETE. For UPDATEs, only the affected indexes
are operated upon. Index maintenance is done in parallel as much as possible (for example,
we pipeline rows from the base table to the indexes, and we update the indexes themselves
in parallel). Of course, all of  this index maintenance is done under a transaction, so we
maintain consistency between indexes and base table.

For SELECTs, the Trafodion optimizer will pick which secondary indexes to access based upon
cost. Several possibilities exist: It may do an index-only access (if all the relevant columns
happen to be available in some secondary index), a base table only access (if no secondary
index is relevant or cheap) or a join of a secondary index to the base table (e.g. if not
all of the relevant columns are in the index, but we can get direct access to the index and
then use those keys to get direct access to the base table).

Hope this helps,

Dave

-----Original Message-----
From: Devaraj Das [mailto:ddas@hortonworks.com]
Sent: Wednesday, June 28, 2017 2:24 PM
To: dev@trafodion.incubator.apache.org
Subject: Secondary index maintenance/management code in Trafodion..

Hi folks, I am curious to know how Trafodion handles secondary-indices' management and maintenance
for data table stored in hbase. Can you please point me to the design/code for such.

Thanks

Devaraj



Mime
View raw message