james-server-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Norman Maurer <norman.mau...@googlemail.com>
Subject Re: [IMAP] Over-designed /Some thoughts ?
Date Wed, 28 Apr 2010 07:44:08 GMT
2010/4/28 Robert Burrell Donkin <robertburrelldonkin@gmail.com>:
> On Wed, Apr 28, 2010 at 5:55 AM, Norman Maurer
> <norman.maurer@googlemail.com> wrote:
>> Hi Robert,
>> nice to see you are back ;)... Comments inline..
> i'm really busy ATM (exams, projects, reports) :-/
> this will probably have to be my last post for a while...

Ok, hope to see you back in some time..

>> 2010/4/27 Robert Burrell Donkin <robertburrelldonkin@gmail.com>:
>>> On Tue, Apr 27, 2010 at 10:42 AM, Norman Maurer <norman@apache.org> wrote:
>>>> Hi all,
>>>> after spending some time over the weekend to fix some issues with IMAP
>>>> I started to feel its a big "over-designed" ...
>>> i started out with that impression. after digging around i came to the
>>> conclusion that IMAP has some annoying requirements..
>> Oh well... its some kind of a pita.
> +1
> sessions that select a mailbox have to be updated by operations done
> by other sessions on that mailbox

Yes I'm aware of this.. I'm still trying to get my head around of a
good solution to see without breakin to much ;)

>>>> 1) UidChangeTracking:
> the API uses a listeners and events to manage updates to UIDs etc.
> this design may need to be revised.

I think the event stuff is good. Just the Tracker is not needed and
add a "useless" layer of complexity to the api..

>>>> 2) Global Mailbox caching
>>>> At the moment the Mailbox is cached in a HashMap. The problem with
>>>> this is it will never get recycled by the GC. This can generate a OOM
>>>> over long time
>>> i run IMAP with approx 1.5G spread over around a hundred mailboxes.
>>> i've never had an OOM. so i never bothered changing this.
>> I think you use Torque right ? Maybe it behave a bit different there.
>> I'm using JPA and its reproducable with feeding a mailbox with ca 1
>> million emails. You will see the memory usage just grow and grow..
>> When I took a heap dump it seems like the OpenJPA objects where never
>> released, because the where hold in the HashMap.
> AFACT the OpenJPA stuff shouldn't be caching the mailboxes in the
> manager (caching should be managed internally by OpenJPA). sounds like
> a session management problem is much more likely. ATM sessions cache
> entity managers for the duration. if you're running a lot of
> concurrent connections, they need to be pooled.

Yeah we should remove the caching of mailboxes at all from the
MailboxManager, because (as you already stated) it depend on the
implementation if a cache is needed and how a cache is implemented.

At the moment I refactored the openjpa stuff to use one EntityManager
per request. This seems to work out well so far. Even better would be
to use one entitymanager per mailboxsession.

> in IMAP, a session may:
> * have a long term interest in a mailbox spanning multiple requests
> * need to perform multiple operations on one or more mailboxes to
> execute a single protocol request
> IMHO the failure to cleanly and clearly separate these is a major flaw
> in current API. the Mailbox caching issues are just a consequence.
> fixing these would require a major rewrite with knowledge of the
> current foibles...
> - robert


Ps: And good luck for your exams mate..

To unsubscribe, e-mail: server-dev-unsubscribe@james.apache.org
For additional commands, e-mail: server-dev-help@james.apache.org

View raw message