james-server-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Ahmed Mohombe <amoho...@yahoo.com>
Subject Re: IMAP Draft: Cluster
Date Fri, 07 Jul 2006 09:23:32 GMT
>>>>> In a worst case, when nobody is ill or on holidays, you have at least
>>>>> one open connection per user all the time. 
>>>> How would one handle in the JVM and code these very long lasting connections?
>>> Why? Are there issues? 
>>> I'm not aware of any problems with many sockets treated by many threads
>>> running a long time apart from the not avoidable network, cpu and memory
>>> consumption. 
>> Well, if too many resources are for long time "occupied" and not released,
>> the JVM won't even get the change to do GC on those object trees. 
> Is it like in Windows 95 that had unavoidable crashed after 21 days? ;-)
> Or like in Windows 98 that had to be rebooted to get back full ;-)
> performance?
Well almost, but but not that bad :).

> Well that would deny using Java on servers at all. You mean if I set a
> variable that was bound for 1 week to null it won't ever by gc'ed?
> A socket that has been opened for 24h can't be closed?
> Well I know many servers, just like James, are fighting with memory
> leaks and class loader issues. But those problems are mostly self-made
> and maybe seldom related to JVM Bugs...
Exactly. This is not the fault of the JVM, nor Java, but of the applications.
However the reality is that complex software *have* bugs. This is a matter
of fact, and in the context of long running it is even more visible.
Nowadays each software depends on so many frameworks/libraries, that is
almost impossible to eliminate all the bugs, and in many cases new bugs can come
in by "innocent updates" of some libs.

> Could you give a concrete example? 
Let's take Tomcat. It's a wonderful example. It has a very high quality,
still many versions have some sort of memory leaks or other bugs that can
be seen for services running over months. Restarting the server every week-end
or every month is a common practice in most of the companies, and seems to
be much cheaper than to search the real bugs (that in many of the cases are
in the custom webapplication, or the combination of webapplication+Tomcat).
I had many cases when simply upgrading to a new Tomcat version the webapplications
didn't exposed more memory leaks (but unfortunately the new tomcat version
brought other new bugs).

AFAIK Mailets are supposed to be used with IMAP too, so the situation
might be a little similar to Tomcat :).


>> The problem is also that those resources are limited (at least for most
>> of the people).
> 
> Memory and cpu, of course. They will limit the possible number of
> threads and sockets. I don't think we will get into limits just because
> of number of threads running or number of sockets opened.
> 
>>> In this case the IMAP server should run on a separate machine not to
>>> influence smtp/spooling on high loads.
>> Having a second machine is already a too high system requirement. Many
>> small companies that have their server to an ISP would gladly use IMAP
>> but only on that one machine.
> 
> Using one machine will always mean that you have to limit performance to
> play safe and have reserves. This will first be done by an obligatory
> connection limit. After having benchmarks results we could see how a
> dynamic connection limit could make sense.
This sounds good.

> But the best performance will be achieved with a dedicated machine. 
I know, but this is what most of the users won't have, nor do they have now.
A big problem with J2EE in general is that that such applications are fantastic
ant very scalable, but only upward (very seldom downward). This is why PHP has
such a big success.
IMHO this downward scalability should not be forgotten in the case of JAMES.


>>>>>> Clustering at DB level would mean to not support dbfile, file and
other 
>>>>>>   types of repositories.
>>>>> Clustering at DB level would just mean following some rules in the
>>>>> implementation. DBMS are designed to be accessed simultaneously from
>>>>> multiple hosts.
>>>>> dbfile mmh, good point. I just made a few thoughts. There are also
>>>>> various solutions to access a network file-system without a single point
>>>>> of failure. 
>>>> What about JCR? (e.g. the Jackrabbit implementation). Could
>>>> it solve some of the problems, or it is too far away from making
>>>> it usable as a "file system" for IMAP?
>>> Well, I don't have any knowledge about JCR. I tried to quickly browse
>>> some docs/faqs. 
>>> But it seems to be able to use it reliable Jackrabbit uses a JDBC
>>> back-end, too. And I guess it wasn't made to store lots of emails. The
>>> overhead of implementing this might be greater than using JDBC directly.
>>> This gives us the ability to tune performance.
>>> How do you think JCR might help?
>> Well, JCR is a standard and can have many types of back-ends (not just JDBC).
> 
> Yes, but the questions is which back-ends are already there for
> production and what are the features. E.g. jackrabbit site says,
> file-based back-end does not support transactions and might by
> inconsistent after an unclean JVM shutdown.
Yes, I know. The Jackrabbit implementation is not perfect, but IMHO when evaluating
if to use JCR or not, the standard should matter as that is ready (the implementation
can be improved till it fully implements the standard).
I'm not an expert in JCR nor IMAP, that's why I asked.
Considering that Jackrabbit is also an Apache project, maybe some of the authors could
better answer some questions.
IMHO is worth asking since if JCR is usable/compatible with IMAP than it could shorten greatly
the 
release date of JAMES IMAP.

>> I asked about JCR since IMAP from a user perspective looks more like a DMS. In fact,
>> many small companies use the Exchange server(and some Outlook plug-ins) with IMAP
exactly in this 
>> way: to access easily all the documents from everywhere and all the time.
> 
> Yeah, another example why imap is so resource intensive. :-)
I really don't know how could one prevent such misuse. It is the typical "hammer" syndrome
:).

Ahmed.


---------------------------------------------------------------------
To unsubscribe, e-mail: server-dev-unsubscribe@james.apache.org
For additional commands, e-mail: server-dev-help@james.apache.org


Mime
View raw message