james-server-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Norman Maurer <norman.mau...@googlemail.com>
Subject Re: [IMAP] Over-designed /Some thoughts ?
Date Wed, 28 Apr 2010 07:44:08 GMT
2010/4/28 Robert Burrell Donkin <robertburrelldonkin@gmail.com>:
> On Wed, Apr 28, 2010 at 5:55 AM, Norman Maurer
> <norman.maurer@googlemail.com> wrote:
>> Hi Robert,
>>
>> nice to see you are back ;)... Comments inline..
>
> i'm really busy ATM (exams, projects, reports) :-/
>
> this will probably have to be my last post for a while...
>

Ok, hope to see you back in some time..

>> 2010/4/27 Robert Burrell Donkin <robertburrelldonkin@gmail.com>:
>>> On Tue, Apr 27, 2010 at 10:42 AM, Norman Maurer <norman@apache.org> wrote:
>>>> Hi all,
>>>>
>>>> after spending some time over the weekend to fix some issues with IMAP
>>>> I started to feel its a big "over-designed" ...
>>>
>>> i started out with that impression. after digging around i came to the
>>> conclusion that IMAP has some annoying requirements..
>>
>> Oh well... its some kind of a pita.
>
> +1
>
> sessions that select a mailbox have to be updated by operations done
> by other sessions on that mailbox

Yes I'm aware of this.. I'm still trying to get my head around of a
good solution to see without breakin to much ;)

>
>>>> 1) UidChangeTracking:
>
> the API uses a listeners and events to manage updates to UIDs etc.
> this design may need to be revised.

I think the event stuff is good. Just the Tracker is not needed and
add a "useless" layer of complexity to the api..

>
>>>> 2) Global Mailbox caching
>>>>
>>>> At the moment the Mailbox is cached in a HashMap. The problem with
>>>> this is it will never get recycled by the GC. This can generate a OOM
>>>> over long time
>>>
>>> i run IMAP with approx 1.5G spread over around a hundred mailboxes.
>>> i've never had an OOM. so i never bothered changing this.
>>
>> I think you use Torque right ? Maybe it behave a bit different there.
>> I'm using JPA and its reproducable with feeding a mailbox with ca 1
>> million emails. You will see the memory usage just grow and grow..
>> When I took a heap dump it seems like the OpenJPA objects where never
>> released, because the where hold in the HashMap.
>
> AFACT the OpenJPA stuff shouldn't be caching the mailboxes in the
> manager (caching should be managed internally by OpenJPA). sounds like
> a session management problem is much more likely. ATM sessions cache
> entity managers for the duration. if you're running a lot of
> concurrent connections, they need to be pooled.
>

Yeah we should remove the caching of mailboxes at all from the
MailboxManager, because (as you already stated) it depend on the
implementation if a cache is needed and how a cache is implemented.

At the moment I refactored the openjpa stuff to use one EntityManager
per request. This seems to work out well so far. Even better would be
to use one entitymanager per mailboxsession.

> FWIW
>
> in IMAP, a session may:
>
> * have a long term interest in a mailbox spanning multiple requests
> * need to perform multiple operations on one or more mailboxes to
> execute a single protocol request
>
> IMHO the failure to cleanly and clearly separate these is a major flaw
> in current API. the Mailbox caching issues are just a consequence.
> fixing these would require a major rewrite with knowledge of the
> current foibles...
>
> - robert

Bye,
Norman

Ps: And good luck for your exams mate..

---------------------------------------------------------------------
To unsubscribe, e-mail: server-dev-unsubscribe@james.apache.org
For additional commands, e-mail: server-dev-help@james.apache.org


Mime
View raw message