hbase-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Doug Meil <doug.m...@explorysmedical.com>
Subject Re: A requirement to change time of the Hbase cluster.
Date Wed, 26 Oct 2011 19:00:25 GMT

+1.  Well stated Gary.

Doing anything else is asking for trouble, and it's preventable.




On 10/26/11 2:39 PM, "Gary Helmling" <ghelmling@gmail.com> wrote:

>At the same time, it might be simpler to get your customers/operators
>to fix their ntp setups.
>
>Not having synchronized clocks throughout the cluster will cause
>problems in other areas as well.  It will make it very difficult to
>correlate events in different server logs when troubleshooting
>problems (you can't rely on the timestamp as a rough guideline).  The
>Kerberos infrastructure used by Hadoop and HBase security also
>requires synchronized clocks to operate correctly.  By default, if the
>clock skew between two machines is greater than 5 minutes, it will
>reject messages as invalid.  So you're likely to experience other
>headaches even if you can coax HBase into operating the way you'd
>like.
>
>
>On Wed, Oct 26, 2011 at 9:02 AM, Stack <stack@duboce.net> wrote:
>> On Tue, Oct 25, 2011 at 7:36 PM, Gaojinchao <gaojinchao@huawei.com>
>>wrote:
>>> So we hope to add a choice about metadata is not time-dependent. Just
>>>like use data can use a number as a timestamp .
>>> If we can do this, the effect for time will be smaller. We don't use
>>>the ntp server, the cluster also can work normal ?
>>> Can I open a file? I will try to make a patch and share my mind.
>>>
>>
>> I suppose you could set an attribute on a table that says "use always
>> increasing version rather than timestamp".  You'd have to then on a
>> per region basis keep note of the most recent version and rather than
>> use system time, do a +1 per edit coming in.
>>
>> I think hfile already records the version of the last edit added to
>> the file.  On open of a region, you'd look at all hfiles and figure
>> the highest verison and then set your version machine to start at
>> highest-version +1.
>>
>> It might not be that hard to add.  You'd have to check the code but a
>> while back we made it so we indirectly got version by going to an
>> EnvironmentEdge class.  You could add your 'always increasing version'
>> as an atomic long or something and then it would be available
>> throughout.
>>
>> St.Ack
>>


Mime
View raw message