hbase-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Nanheng Wu <nanhen...@gmail.com>
Subject Re: Disabling a table taking very long time
Date Fri, 25 Feb 2011 07:37:57 GMT
I think you are right, maybe in the long run I need to re-architect my
system so that it doesn't need to create new and delete old tables all
the time. In the short term I am having a really hard time with the
disabling function, I ran a disable command on a very small table
(probably dozen of MBs in size) and are no client using the cluster at
all, and that took about 1 hour to complete! The weird thing is on the
web UI only the region server carrying the META table has non-zero
requests, all other RS have 0 requests the entire time. I would think
they should get some request to flush the memstore at least. I *am*
using the same RS nodes for some map reduce job at the time and top
shows the memory usage is almost full on the META region. Would you
have some idea of what I should investigate?
Thanks so much.

On Thu, Feb 24, 2011 at 3:24 PM, Jean-Daniel Cryans <jdcryans@apache.org> wrote:
> I haven't tried, but it seems incredibly hacky and bound to generate
> more problems than it solves. Instead you could consider using
> different table names.
>
> J-D
>
> On Thu, Feb 24, 2011 at 3:21 PM, Nanheng Wu <nanhengwu@gmail.com> wrote:
>> What would happen if I try to remove the region files from hdfs and
>> then remove the regions from the meta table? Is hbase gonna get messed
>> up when it later needs to flush those regions out of the memstore?
>> Thanks!
>>
>> On Thursday, February 24, 2011, Jean-Daniel Cryans <jdcryans@apache.org> wrote:
>>> Exactly.
>>>
>>> J-D
>>>
>>> On Thu, Feb 24, 2011 at 2:45 PM, Nanheng Wu <nanhengwu@gmail.com> wrote:
>>>> Sorry for trying to bring this topic back again guys, so currently in
>>>> 0.20.6 is there's no way to drop a table without large amount of
>>>> flushing?
>>>>
>>>> On Tue, Feb 22, 2011 at 3:04 PM, Jean-Daniel Cryans <jdcryans@apache.org>
wrote:
>>>>> Moving this discussion to jira: https://issues.apache.org/jira/browse/HBASE-3557
>>>>>
>>>>> Good stuff guys.
>>>>>
>>>>> J-D
>>>>>
>>>>> On Mon, Feb 21, 2011 at 8:25 AM, Hari Sreekumar
>>>>> <hsreekumar@clickable.com> wrote:
>>>>>> I agree, and I have missed this a lot too. We should have a way to
directly
>>>>>> drop the table, it would help save a lot of time while developing
/testing.
>>>>>> Maybe also have a config entry to enable this behavior, like we have
>>>>>> webinterface.private.actions in hadoop. It can be enabled on the
dev cluster
>>>>>> and disabled on production etc.
>>>>>>
>>>>>> Hari
>>>>>>
>>>>>> On Mon, Feb 21, 2011 at 12:41 PM, Lars George <lars.george@gmail.com>
wrote:
>>>>>>
>>>>>>> Could be having some "force" flag specified 3 times and asked
for
>>>>>>> confirmation as well, but I like this feature. Whenever I talk
to
>>>>>>> people who disable and get stuck it was to prepare a subsequent
drop
>>>>>>> table call. So this sounds really useful given enough safety
latches
>>>>>>> in place.
>>>>>>>
>>>>>>> Lars
>>>>>>>
>>>>>>> On Thu, Feb 17, 2011 at 1:50 AM, Jean-Daniel Cryans <jdcryans@apache.org>
>>>>>>> wrote:
>>>>>>> > Actually I never thought of having a special case for that...
and I
>>>>>>> > don't see any jira about it. Would you mind opening a new
one for
>>>>>>> > that, I think it's a good idea for those times when you're
developing
>>>>>>> > something and you want to iterate fast.
>>>>>>> >
>>>>>>> > On the other hand, it's a pretty destructive feature so
some people
>>>>>>> > might disagree with having it in the codebase :)
>>>>>>> >
>>>>>>> > J-D
>>>>>>> >
>>>>>>> > On Wed, Feb 16, 2011 at 4:26 PM, Nanheng Wu <nanhengwu@gmail.com>
wrote:
>>>>>>> >> Actually I wanted to disable the table so I can drop
it. It would be
>>>>>>> >> nice to be able to disable the table without flushing
memstore. It's
>>>>>>> >> not possible in 0.20.6 is it?
>>>>>>> >>
>>>>>>> >> On Wed, Feb 16, 2011 at 2:30 PM, Jean-Daniel Cryans
<
>>>>>>> jdcryans@apache.org> wrote:
>>>>>>> >>> To disable a region, it's memstore must first be
flushed. That's done
>>>>>>> >>> sequentially for all regions that must be disabled
inside a table.
>>>>>>> >>> There's not really a way around it unless you don't
need that data.
>>>>>>> >>>
>>>>>>> >>> If you're planning to disable a table but at the
same time need it to
>>>>>>> >>> be unavailable for the shortest time possible, I
recommend calling a
>>>>>>> >>> flush on the table from the shell first and then
some time later doing
>>>>>>> >>> the disable. How much later you ask? Well there's
currently no easy
>>>>>>> >>> way to tell, I usually just tail any region server
log file until I
>>>>>>> >>> see they're done.
>>>>>>> >>>
>>>>>>> >>> J-D
>>>>>>> >>>
>>>>>>> >>> On Wed, Feb 16, 2011 at 2:21 PM, Nanheng Wu <nanhengwu@gmail.com>
>>>>>>> wrote:
>>>>>>> >>>> From time to time I run into issues where disabling
a table pretty
>>>>>>>
>>
>

Mime
View raw message