james-server-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Shilpa Dalmia <Shi...@PostX.com>
Subject RE: writing mails to the spool store in between processors
Date Thu, 09 May 2002 17:53:39 GMT
I'm putting down a step by step listing of the proposed solution with the
effect it will have on current processing.

Problem we're trying to solve
1. Avoid having to rename the mail message everytime the message is being
written back to the spool to be transferred to the next processor. 
2. Improve performance by minimizing disk I/O for disk spooling.

Solution 

Part 1:

Do not rename the message before writing to the spool. Remove the message
only if it has finished processing (check the state is GHOST or no
recipients to be processed).

Advantage:
1. Reduce disk i/o by not having to rewrite the same message again to the
disk, just writing the header is enough.
2. Does not affect too many files. The change has to be made only in
LinearProcessor.java (remove the renaming of mail message).
I've tested this solution & it gives a 40% boost in the performance.

snippet of service method() in LinearProcessor (changes highlighted in
bold.)

                //mail.setName(newName(mail));
                spool.store(mail);
                mail = null;
                continue;
            } else {
                //Ok, we made it through with the same state... move it to
the next
                //  spot in the array
                unprocessed[i + 1].add(mail);
            }


and
JamesSpoolManager.java (check if the mail is in GHOST state or has no more
recipients, before removing it)

                   if (mail.getState().equals(mail.GHOST) ||
                    mail.getRecipients() == null ||
mail.getRecipients().size() == 0)
                    {
                      spool.remove(key);
                      getLogger().info("==== Removed from spool mail "
                                       + mail.getName() + " ====");
                    }
                    mail = null;
 
Part 2:

We could further improve the performance (minimize disk i/o) by caching the
message when it is first written to the spool. I've created a new protocol
"filemem" based on the existing "file" protocol which uses both disk
spooling and memory spoling. Create a memory spool repository & write the
spool message to this spool after it has been written to the disk spool. All
subsequent spool accesses, by different processors, would be through the
memory spool. Once processing is done, message will be removed from both the
spools. Disk spool will be used for retries.

I have implemented & tested these solutions. If you think it is worth
incorporating it in the James code base, I'd be happy to do that.

Shilpa Dalmia
PostX Corporation

--
To unsubscribe, e-mail:   <mailto:james-dev-unsubscribe@jakarta.apache.org>
For additional commands, e-mail: <mailto:james-dev-help@jakarta.apache.org>


Mime
View raw message