struts-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Gavin Terrill" <gterr...@karora.ca>
Subject RE: When to store files in DB vs. File system
Date Mon, 03 Dec 2001 16:03:25 GMT
I would recommend using the file system, and storing the location in the
db:
- the file system is faster
- storing files in the DB tends to generate enormous database physical
files, so back ups become an issue
- integrating with a freetext search engine becomes harder (you may need
this down the track)

We have not seen issues with the number of files in a directory,
although we also partition files across directories to minimize the
number of files and avoid OS limitations (a simple way to do this is to
use YYYY\MM\DD as the directory structure).

<vendor>
You may also want to check out: http://www.karora.com/DocConn.htm
</vendor>

Gavin Terrill
CTO, Karora Technologies Inc.
http://www.karora.com



-----Original Message-----
From: Ted Husted [mailto:husted@apache.org] 
Sent: Monday, December 03, 2001 6:48 AM
To: Struts Users Mailing List
Subject: Re: When to store files in DB vs. File system


I've seen a lot of implementations do this sort of thing, especially
when archive emails and the like.

I've also heard that people find many OS's start to slow down with even
1000 files in a directory, so I believe many such systems use that as an
increment /1/000 - /1/999.

A general piece of good advice is that databases excel at storing
itty-bitty pieces of information, and files systems are good for storing
a larger blocks. If you decouple the actual files from the rest of the
database, you also have the opportunity to centralize that stored on a
machine optimized for that purpose. 

If you need something to index those files, I'd have a look at Lucene
;-)

http://jakarta.apache.org/lucene


-- Ted Husted, Husted dot Com, Fairport NY USA.
-- Custom Software ~ Technical Services.
-- Tel +1 716 737-3463
-- http://www.husted.com/struts/

Darryl Nortje wrote:
> 
> Hi.
> 
> Everyone is going to tell you that you must store it in the database. 
> The reason are that in different OS's, there is a limit to how many 
> files you can store in each directory. (windows i think is 10000). I 
> was faced with this same problem under a time constraint, so never 
> took the time to figure out how to store the file directly into the 
> database. So i went the a directory storing route. What I did was 
> Create a directory root\1\1a\1aa\file.extension like this. Set a limit

> to the amount of files in this directory and when that limit was 
> reached incremented the directory name to root\1\1a\1ab etc etc. And 
> then just stored the directory and filename in the database and 
> retrieved it like that. I'm pretty sure that this is not the correct 
> way to do this sort of thing, but it does work. I will be watching 
> this mail to find out how to do it properly.
> 
> -----Original Message-----
> From: Andy Timm [mailto:java_coder@yahoo.com]
> Sent: Monday, December 03, 2001 9:53 AM
> To: struts-user@jakarta.apache.org
> Subject: When to store files in DB vs. File system
> 
> Hello, I am developing a webapp where users need to
> store files (<200k).  I am wondering if these should
> be stored in the (mysql) database, or in the file
> system.  I am concerned about the number of files I
> could get in the database (possibility for 20,000+
> users, so 100,000 files).  On the other hand, I plan
> on replicating my database across multiple servers.
> Do I then also replicate the download of each file, or
> run a batch that copies them periodically?  I would
> greatly appreciate advice from anyone who has dealt
> with these design issues.  Thank you, Andrew Timm
> 
> __________________________________________________
> Do You Yahoo!?
> Buy the perfect holiday gifts at Yahoo! Shopping. 
> http://shopping.yahoo.com
> 
> --
> To unsubscribe, e-mail: 
> <mailto:struts-user-unsubscribe@jakarta.apache.org>
> For additional commands, e-mail: 
> <mailto:struts-user-help@jakarta.apache.org>
> 
> --
> To unsubscribe, e-mail:
<mailto:struts-user-unsubscribe@jakarta.apache.org>
> For additional commands, e-mail: 
> <mailto:struts-user-help@jakarta.apache.org>

--
To unsubscribe, e-mail:
<mailto:struts-user-unsubscribe@jakarta.apache.org>
For additional commands, e-mail:
<mailto:struts-user-help@jakarta.apache.org>


--
To unsubscribe, e-mail:   <mailto:struts-user-unsubscribe@jakarta.apache.org>
For additional commands, e-mail: <mailto:struts-user-help@jakarta.apache.org>


Mime
View raw message