james-server-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Stefano Bagnara <apa...@bago.org>
Subject MOM and spooling (Was: Geronimo-JAMES integration)
Date Tue, 24 Jun 2008 18:40:46 GMT
Robert Burrell Donkin ha scritto:
> On Mon, Jun 16, 2008 at 8:37 PM, Stefano Bagnara <apache@bago.org> wrote:
>> Robert Burrell Donkin ha scritto:
>>> On Mon, Jun 16, 2008 at 1:01 PM, Stefano Bagnara <apache@bago.org> wrote:
>>>> Robert Burrell Donkin ha scritto:
>>>>> generally: we've talked before and the whole design around that area
>>>>> of the system could be so much better
>>>>>
>>>>> technology: actually support for any distributable messaging solution
>>>>> but i can see no reason not to use a standard
>>>> I agree.
>>>>
>>>> FYI I had an interesting discussion with James Strachan few weeks ago,
>>>> here:
>>>> http://markmail.org/message/mp2pafe77efwczbb
>>> it's always interesting talking to james :-)
>>>
>>> he's definitely thinking in the same directions as me
>>>
>>> - robert
>> I had no time yet to check CAMEL, and there are few issues we already found
>> discussing the JMS solution (no way to avoid writing the full message over
>> and over again when moving from one queue to another is my main concern),
>> but I bookmarked the thread and camel for my next "meditation" weekend
>> (unfortunately not so frequent lately).
> 
> MOM works surprisingly well provided that the message sizes are small.
> IMHO JAMES should not be attempting to load large bodies into memory.
> instead, a limited buffer should be used initially: big enough to hold
> the headers (and small messages). once this is filled, the message
> should be streamed directly to storage. the message would contain the
> headers plus a reference URL for the body.

About my concern it still applies also to the scenario where only the 
"Envelope" (to be defined) is written on every queue change, but this 
cannot be avoided with the suggested approach unless we switch to 
non-persistent queues.

Sometimes my JAMES deal with around a million email in a day and I had 
to "hack" some JAMES code to avoid writing to disk/jdbc on each status 
change (custom processorClass for the spoolmanager).

A performance oriented approach would be to directly use the KAHA engine 
  [1] (or maybe the AMQ Message Stores [2]) o to write stuff only for 
reliability but keep using memory operations unless the queue grow too 
much, but I think this should be kept in our mind when we'll refactor 
the repository APIs (let's not close this door when we change the api)

It would be cool if a standard SMTP=>MultipleMailets=>RemoteDelivery 
processing could be done with a single write to disk when the mail is 
accepted and no other read/write (unless memory/queues are full).

Stefano

[1] http://activemq.apache.org/kaha-persistence.html
[2] http://activemq.apache.org/amq-message-store.html


---------------------------------------------------------------------
To unsubscribe, e-mail: server-dev-unsubscribe@james.apache.org
For additional commands, e-mail: server-dev-help@james.apache.org


Mime
View raw message