mina-ftpserver-users mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Brett M. Bergquist" <br...@thebergquistfamily.com>
Subject Re: LIST from database
Date Tue, 27 Apr 2010 21:13:31 GMT
I just did a quick modification of my code to not do a select on each 
file. Basically when the "list" is done, I gather all of the information 
and cache it in the DbFile object.  This greatly increased performance 
so that at directory with a couple of hundred files takes less than a 
second to return a directory listing.

I have created a Jira issue and attached a Netbeans project which is a 
running sample against a Derby database.  This has this changed code in 
it.  So you might want to download the code and make the same changes in 
your version.

This is Jira Issue: FTPSERVER-372 </jira/browse/FTPSERVER-372>

https://issues.apache.org/jira/browse/FTPSERVER-372


Brett M. Bergquist wrote:
> Sorry, I just saw this among the hundreds of emails that I get ;)
>
> I don't have a quick answer.  It is doing 1 select to get the list of 
> files and then for each other check on each file (isDirectory, 
> isReadable, etc.) it is doing another select.  It is a very naive 
> implementation in that it is caching nothing and always refreshing its 
> knowledge from the database.   And your right, with many files in a 
> directory it is slow and resource intensive.
>
> Kenneth Vanvik Hansen wrote:
>> No, i don't have a need to do a select for every file. It just seems 
>> it does
>> a lot of queries when it receives a LIST command. Only change I have 
>> done to
>> the original code is I have changed from Derby to ODBC. It works 
>> great for
>> folders with up to 10 or so files in it, after that it starts to get 
>> really
>> slow.
>>
>> -----Original Message-----
>> From: David Latorre [mailto:dvlato@gmail.com] Sent: 27. april 2010 14:10
>> To: ftpserver-users
>> Subject: Re: LIST from database
>>
>> If I inderstood you correctly, it is your code that is doing a SELECT 
>> for
>> every file, isn't it?
>>
>> Since I haven't seen your code, I cannot give you advice on this, why 
>> do you
>> need to do a select for every file?
>>
>> Maybe Brett's solution can help you... did you get the time to work 
>> on this,
>> Brett?
>>
>>
>>
>>
>> 2010/4/27 Kenneth Vanvik Hansen <kvhansen@online.no>:
>>  
>>> Hi, I've been working on this a bit now but I'm having some trouble 
>>> when there are many files/folders in a folder. Seems like LIST does 
>>> a select for every file in a folder. Any good ideas on how to solve 
>>> this? In my db(which is a bit slow) it can take several minutes to list
>>>     
>> 40+ files/folders.
>>  
>>> Kenneth
>>>
>>> -----Original Message-----
>>> From: Brett M. Bergquist [mailto:brett@thebergquistfamily.com]
>>> Sent: 23. mars 2010 17:40
>>> To: ftpserver-users@mina.apache.org
>>> Subject: Re: LIST from database
>>>
>>> Okay.  That will work.  Thanks.
>>>
>>> Niklas Gustavsson wrote:
>>>    
>>>> On Tue, Mar 23, 2010 at 4:08 PM, Brett M. Bergquist 
>>>> <brett@thebergquistfamily.com> wrote:
>>>>
>>>>      
>>>>> I actually have a working Netbeans project that works against a 
>>>>> Derby database.  I am going to do a little more cleanup and see if 
>>>>> I cannot factor out the database specifics like the db user 
>>>>> manager does and then I would like to provide the whole thing, 
>>>>> probably to go into the examples.  So should I send this to you 
>>>>> Niklas when I am
>>>>>         
>> ready?
>>  
>>>> If you would like to contribute the code to the project, the best 
>>>> way is to create a JIRA issue and attach the code there (making 
>>>> sure you check the donate to ASF checkbox).
>>>>
>>>> /niklas
>>>>
>>>>
>>>>       
>>>
>>>     
>>
>>
>>
>>   
>

Mime
View raw message