hbase-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Ted <yuzhih...@gmail.com>
Subject Re: CleanerChore exception
Date Mon, 31 Dec 2012 00:22:35 GMT
I am not at a computer for the moment. 
This involves passing an extra parameter indicating the level of recursion and possibly using
an enum in place of the Boolean return value. 
I will show you a patch when I get home. 

Thanks

On Dec 30, 2012, at 2:42 PM, Jean-Marc Spaggiari <jean-marc@spaggiari.org> wrote:

> I'm not sure I'm getting that.
> 
> It's recursive. So when you are on the parent directory, you don't
> know yet if the child directory is empty or not. So you can't call the
> delete() yet. If you call the delet() giving "true" for recurs, then
> you might delete some files who just got created, which we want to
> avoid.
> 
> IMHO.
> 
> 2012/12/30, Ted <yuzhihong@gmail.com>:
>> Thanks for your digging.
>> 
>> Minor optimization would be to issue delete() on the parent directory so
>> that there are fewer requests to namenode.
>> 
>> Cheers
>> 
>> On Dec 30, 2012, at 2:15 PM, Jean-Marc Spaggiari <jean-marc@spaggiari.org>
>> wrote:
>> 
>>> I did the change, pushed it and it cleaned my directories correctly.
>>> 
>>> // if the directory doesn't exist or is empty, then we are done
>>> if (children == null) return fs.delete(toCheck, false);
>>> 
>>> The only thing is that I don't know what will fs.delete() return i
>>> case the directory doesn't exist. But I think it's still correct to
>>> return false if the directory doesn't exist because we can't really
>>> delete something which doesn't exist...
>>> 
>>> My opinion.
>>> 
>>> So the patch is ready, easy one ;) Just waiting for Jesse's feedback
>>> just in case.
>>> 
>>> JM
>>> 
>>> 2012/12/30, lars hofhansl <lhofhansl@yahoo.com>:
>>>> Nothing has changed around this in 0.94.4 as far as I know.
>>>> 
>>>> 
>>>> 
>>>> 
>>>> ________________________________
>>>> From: Jean-Marc Spaggiari <jean-marc@spaggiari.org>
>>>> To: user@hbase.apache.org
>>>> Sent: Sunday, December 30, 2012 9:53 AM
>>>> Subject: Re: CleanerChore exception
>>>> 
>>>> I was going to move to 0.94.4 today ;) And yes I'm using 0.94.3. I
>>>> might wait a bit in case some testing is required with my version.
>>>> 
>>>> Is this what you are looking for? http://pastebin.com/N8Q0FMba
>>>> 
>>>> I will keep the files for now since it seems it's not causing any
>>>> major issue. That will allow some more testing if required.
>>>> 
>>>> JM
>>>> 
>>>> 
>>>> 2012/12/30, Ted Yu <yuzhihong@gmail.com>:
>>>>> Looks like you're using 0.94.3
>>>>> 
>>>>> The archiver is backport of:
>>>>> HBASE-5547, Don't delete HFiles in backup mode
>>>>> 
>>>>> Can you provide more the log where the IOE was reported using pastebin
>>>>> ?
>>>>> 
>>>>> Thanks
>>>>> 
>>>>> On Sun, Dec 30, 2012 at 9:08 AM, Jean-Marc Spaggiari <
>>>>> jean-marc@spaggiari.org> wrote:
>>>>> 
>>>>>> Hi,
>>>>>> 
>>>>>> I have a "IOException" /hbase/.archive/table_name is non empty
>>>>>> exception every minute on my logs.
>>>>>> 
>>>>>> There is 30 directories under this directory. the main directory
is
>>>>>> from yesterday, but all sub directories are from December 10th, all
>>>>>> the same time.
>>>>>> 
>>>>>> What does this .archive directory is used for, and what should I
do?
>>>>>> 
>>>>>> Thanks,
>>>>>> 
>>>>>> JM
>> 

Mime
View raw message