hbase-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From yavuz gokirmak <ygokir...@gmail.com>
Subject Change data capture tool for hbase
Date Mon, 03 Jun 2013 09:16:20 GMT
Hi all,

Currently we are working on a hbase change data capture (CDC) tool. I want
to share our ideas and continue development according to your feedback.

As you know CDC tools are used for tracking the data changes and take
actions according to these changes[1].  For example in relational
databases, CDC tools are mainly used for replication. You can replicate
your source system continuously to another location or db using CDC tool.So
whenever an insert/update/delete is done on the source system, you can
reflect the same operation to the replicated environment.

As I've said, we are working on a CDC tool that can track changes on a
hbase table and reflect those changes to any other system in real-time.

What we are trying to implement the tool in a way that he will behave as a
slave cluster. So if we enable master-master replication in the source
system, we expect to get all changes and act accordingly. Once the proof of
concept cdc tool is implemented ( we need one week ) we will convert it to
a flume source. So using it as a flume source we can direct data changes to
any destination (sink)

This is just a summary.
Please write your feedback and comments.

Do you know any tool similar to this proposal?


1- http://en.wikipedia.org/wiki/Change_data_capture

  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message