hbase-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Stuti Awasthi <stutiawas...@hcl.com>
Subject RE: Custom timestamps
Date Thu, 20 Oct 2011 04:59:47 GMT
Hi St. Ack , Ben

I also have a scenario that in my case I have to take periodical backup of Hbase data. For
that I have will be using export/import tool. I have decided to take backup based on time
range interval. I have read it in some other posts also that it is not good idea for one to
use timestamp field of Hbase.
Till now my POC works fine at my end but I want to know that if I put same scenario in production
will there be any issues to worry ?

Please Suggest.

-----Original Message-----
From: saint.ack@gmail.com [mailto:saint.ack@gmail.com] On Behalf Of Stack
Sent: Thursday, October 20, 2011 3:54 AM
To: user@hbase.apache.org; Ben West
Subject: Re: Custom timestamps

On Wed, Oct 19, 2011 at 12:18 PM, Ben West <bwsithspawn00@yahoo.com> wrote:
> We're storing timestamped data in HBase; from lurking on the mailing list it seems like
the recommendation is usually to make the timestamp part of the row key. I'm curious why this
is - is scanning over rows more efficient than scanning over timestamps within a cell?
>

I'd be surprised if a noticeable difference.

It depends on how you are to access the data.  In the tsdb case for instance, it wants to
get all metrics within a particular time range.
If the timestamp it used were that of the hbase system, then you'd have to do a full table
scan each time to find metrics that had been fired during a particular time period -- i.e.
you'd check each row and see if any entries on the row for the time period you are interested
in -- whereas if the timestamp part of the row key, you instead just have to start scanning
at the opening of the time range you are querying about.


> The book says: "the version timestamp is internally by HBase for things like time-to-live
calculations. It's usually best to avoid setting this timestamp yourself. Prefer using a separate
timestamp attribute of the row, or have the timestamp a part of the rowkey, or both." I understand
that TTL would be ruined (or saved, depending on your goal) by custom timestamps, and I also
gather that the way HBase handles concurrency is through MVCC. But we are using application
level locks, and HBase's TTL functionality applying is a bonus if anything.
>

The books advice errs on the side of being conservative I'd say.

The MVCC that we do internally does not use the cell timestamp but instead a different running
sequence number that is associated
(internally) with cells (I've not heard of an application atop hbase using the hbase timestamps
to do MVCC at the application level).

The locks you talk of, are these the locks provided in hbase HTable API?  If so, are you aware
they are dangerous (see back in this mailing list for explaination)?

> So is there any reason why we shouldn't set the timestamps manually?
>

Generally, hbase works fine with user set timestamps; there can be issues ordering edits if
clients have divergent clocks and the version being set is time-based but I'm probably not
telling you something you don't already know.

St.Ack

::DISCLAIMER::
-----------------------------------------------------------------------------------------------------------------------

The contents of this e-mail and any attachment(s) are confidential and intended for the named
recipient(s) only.
It shall not attach any liability on the originator or HCL or its affiliates. Any views or
opinions presented in
this email are solely those of the author and may not necessarily reflect the opinions of
HCL or its affiliates.
Any form of reproduction, dissemination, copying, disclosure, modification, distribution and
/ or publication of
this message without the prior written consent of the author of this e-mail is strictly prohibited.
If you have
received this email in error please delete it and notify the sender immediately. Before opening
any mail and
attachments please check them for viruses and defect.

-----------------------------------------------------------------------------------------------------------------------

Mime
View raw message