james-server-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Shilpa Dalmia <Shi...@PostX.com>
Subject RE: writing mails to the spool store in between processors
Date Thu, 09 May 2002 23:17:23 GMT
Serge,
   Do u have a timeframe when the avalon upgrade would be done? And what is
the procedure on submitting the changes? Do i just send you the files with
the changes?

Shilpa

-----Original Message-----
From: Serge Knystautas [mailto:sergek@lokitech.com]
Sent: Thursday, May 09, 2002 3:51 PM
To: James Developers List
Subject: Re: writing mails to the spool store in between processors


Shilpa,

Both sound like good ideas... we're about to do an update to the latest 
Avalon, so if you could wait until that's complete before you send any 
diffs, that would be great.

Also, I'd be just as interested in what tests you've setup... we could 
really use those as well.
-- 
Serge Knystautas
Loki Technologies - Unstoppable Websites
http://www.lokitech.com/

Shilpa Dalmia wrote:
> I'm putting down a step by step listing of the proposed solution with the
> effect it will have on current processing.
> 
> Problem we're trying to solve
> 1. Avoid having to rename the mail message everytime the message is being
> written back to the spool to be transferred to the next processor. 
> 2. Improve performance by minimizing disk I/O for disk spooling.
> 
> Solution 
> 
> Part 1:
> 
> Do not rename the message before writing to the spool. Remove the message
> only if it has finished processing (check the state is GHOST or no
> recipients to be processed).
> 
> Advantage:
> 1. Reduce disk i/o by not having to rewrite the same message again to the
> disk, just writing the header is enough.
> 2. Does not affect too many files. The change has to be made only in
> LinearProcessor.java (remove the renaming of mail message).
> I've tested this solution & it gives a 40% boost in the performance.
> 
> snippet of service method() in LinearProcessor (changes highlighted in
> bold.)
> 
>                 //mail.setName(newName(mail));
>                 spool.store(mail);
>                 mail = null;
>                 continue;
>             } else {
>                 //Ok, we made it through with the same state... move it to
> the next
>                 //  spot in the array
>                 unprocessed[i + 1].add(mail);
>             }
> 
> 
> and
> JamesSpoolManager.java (check if the mail is in GHOST state or has no more
> recipients, before removing it)
> 
>                    if (mail.getState().equals(mail.GHOST) ||
>                     mail.getRecipients() == null ||
> mail.getRecipients().size() == 0)
>                     {
>                       spool.remove(key);
>                       getLogger().info("==== Removed from spool mail "
>                                        + mail.getName() + " ====");
>                     }
>                     mail = null;
>  
> Part 2:
> 
> We could further improve the performance (minimize disk i/o) by caching
the
> message when it is first written to the spool. I've created a new protocol
> "filemem" based on the existing "file" protocol which uses both disk
> spooling and memory spoling. Create a memory spool repository & write the
> spool message to this spool after it has been written to the disk spool.
All
> subsequent spool accesses, by different processors, would be through the
> memory spool. Once processing is done, message will be removed from both
the
> spools. Disk spool will be used for retries.
> 
> I have implemented & tested these solutions. If you think it is worth
> incorporating it in the James code base, I'd be happy to do that.
> 
> Shilpa Dalmia
> PostX Corporation


--
To unsubscribe, e-mail:   <mailto:james-dev-unsubscribe@jakarta.apache.org>
For additional commands, e-mail: <mailto:james-dev-help@jakarta.apache.org>

--
To unsubscribe, e-mail:   <mailto:james-dev-unsubscribe@jakarta.apache.org>
For additional commands, e-mail: <mailto:james-dev-help@jakarta.apache.org>


Mime
View raw message