james-server-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Robert Burrell Donkin" <robertburrelldon...@gmail.com>
Subject Re: CFP open for ApacheCon Europe 2009
Date Sun, 02 Nov 2008 07:53:57 GMT
On Sun, Nov 2, 2008 at 12:38 AM, David Jencks <david_jencks@yahoo.com> wrote:
>
> On Nov 1, 2008, at 1:38 PM, Robert Burrell Donkin wrote:
>
>> On Fri, Oct 24, 2008 at 6:56 AM, David Jencks <david_jencks@yahoo.com>
>> wrote:

<snip>

>>> it seems to me that the main reason
>>> you'd be running james inside geronimo would be if your ee apps needed to
>>> send or receive mail.  It seems to me that there should be no need to go
>>> over tcp if both ends are running in the same jvm., delivery ought to be
>>> calling a method directly on the mail server.
>>
>> i can see several potential enterprise use cases
>>
>> 1. shared data access - share data from a standard mail server in an
>> enterprise application. an OpenJPA based data storage layer for james
>> might take care of this.
>> 2. inserting mail into spool - a JEE application wants to deliver mail
>> as if from SMTP or FetchPOP without the overhead and inconvenience of
>> using the protocol. this needs to be done asynchronously. i think that
>> JMS might be a useful candidate. it took me about a day to integrate
>> James with ActiveMQ well enough to allow messages to be delivered into
>> the spool by JMS.
>> 3. direct integration with mailboxes - a JEE application might want to
>> put a mail directly into a mailbox for later retreval by POP3 or IMAP.
>> JCA?
>> 4. triggering JEE processing from a mailet. a mail is accepted into
>> the spool (over FetchPOP or SMTP) and processing procedes. some mail
>> (for example, not SPAM for user joe) should then be passed to a JEE
>> application for processing. again, JMS seems like the right approach
>> to me. again, i had ActiveMQ integration working within a day so this
>> is definitely possible.
>
> I think that activemq vm transport would give the in-vm features I was
> thinking of.  I do wonder if this would introduce either unreliability (jms
> non-persistent messaging) or overhead of redundant storage copies (jms
> persistent messaging, followed by james saving the message).

(i will reach my point eventually but i need to diverge into mail
server architecture for a litttle while...)

most mail servers - including james - have significant storage
overhead. we've talked before about ways to improve this but it's not
an easy topic.

here's my understanding of the current process (hopefully the experts
will jump in and correct my misunderstandings)

when a mail is accepted, it's stored in the spool. later (as resources
allow) a processor picks up the mail from the spool, loads it from
storage and begins the delivery process. james uses matches (rules)
and mailets (actions) to process the mail. to deliver the mail to a
local mailbox, a mailet copies the mail in memory and then stores it
into the mailbox storage.

this may sound inefficient - and you'd be right - but James performs
well in practice and - with some tuning - though the mail latency is
low, throughput is high.

ATM the default James spool manager uses a scanning architecture. it
uses a thread pool and when a thread is free, it scans spool storage
for new mail. if there is new, unprocessed mail, the thread starts
processing one of them.

the experimental ActiveMQ integration was about looking at improving
efficiency, load balancing and separation of permissions by using
ActiveMQ persistent storage as an alternative to the original spool
storage step. mail is accepted by posting to a queue. an
ActiveMQ-based spool manager would use the container for thread
management. mail would arrive as a message to be processed and no
scanning or thread management would need to be performed by the
manager.

i think that using a JMS based spool manager would be the most natural
way to integrate with geronimo

(...back towards the point...)

in terms of efficient embedded (in-vm) usage, direct insertion into
the processing queue will save two copies and one store over the
direct protocol approach.

it would also be possible to use the JEE calling thread to run the
processing code directly. this would save the JMS call and one store.
i have concerns about rollback with this approach. the current mailet
specification is not transactional. providing integrated JPA is used
for the data storage, i think that this should roll back ok but other
mailets may not (for example forwarders etc). providing that this is
made clear, perhaps this isn't a major problem.

> I don't think outbound j2ca is generally appropriate for situations where connections
are free (i.e. in-vm)

(whether connections are really free to a mail server is a moot point.
liveliness is very important when running protocol handlers.
processing throughput therefore needs to be throttled.)

but i was wondering more about providing a single unified contract
which would abstract the complexity of the possible distribution
options from the program into the deployment configuration

- robert

---------------------------------------------------------------------
To unsubscribe, e-mail: server-dev-unsubscribe@james.apache.org
For additional commands, e-mail: server-dev-help@james.apache.org


Mime
View raw message