james-server-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Gabor Kincses <npure2...@yahoo.com>
Subject Re: Disabling the spool in James 2.2.0?
Date Sat, 15 Jan 2005 03:02:29 GMT
Here are results from a very contrived configuration. 
I'll attempt measurement with a more meaningful
configuration later.

After replacing the file-based spool with an in-memory
spool implementation and eliminating a per-message
WARN and an INFO log-message and dropping the hidden
PostMasterAlias-mailet, the performance went to
~16,600 messages/min.  At this point james and the
client split the CPU evenly, indicating that james
probably would have gone up to ~30,000 messages/min on
a 3GHz P4, Win2k, JDK1.5.0_01, NTFS with a message
size of 10.9kB.

I'll attach my changes if there's any interest.

Initially, there were stability issues, since zero
file i/o lead to lock-ups (of james _and_ Windows,
thread priorities?), which I countered by introducing
a forced wake-up from wait() calls every 100
milliseconds and forcing a Thread.yield() after each
notify() call for good measure (the yield() probably
would have been sufficient, I just dread the missed
notifies syndrome). 

BTW, I don't really like wait()/notify(), I much
prefer Doug Lea's synchronization primitives,
LinkedQueue would probably work here.

The in-memory spool repository does not interfere with
the other mail repositories, since the
AvalonMailRepository class was not changed; all the
change is in AvalonSpoolRepository.  One caveat: spool
recovery is of course meaningless after a crash.

No time to profile, yet,
Gabor

--- Gabor Kincses <npure2001@yahoo.com> wrote:

> After replacing the one-byte at a time copy with 1k,
> 4k and 4k+commons-io's ByteArrayOutputStream, the
> performance rose to ~7300 messages/min.  The various
> settings didn't make much difference, not too
> surprisignly since the test messages are 10.9kB a
> piece.
> 
> Time to profile.
> 
> Hope this helps,
> Gabor
> 
> --- Gabor Kincses <npure2001@yahoo.com> wrote:
> 
> > Ok.  Attached is the new version using a byte[] to
> > store the mime message.  The other change is in
> > MailImpl.java to instantiate the new class. 
> > Performance went from ~5500 10.9kB messages/min to
> > ~6800 messages/min.
> > 
> > --- Gabor Kincses <npure2001@yahoo.com> wrote:
> > 
> > > Why?  Is it the synchronization overhead?
> > > 
> > > Which one should I use?  I'm not finding
> anything
> > > other than the java.io.
> > > 
> > > Thanks,
> > > Gabor
> > > 
> > > --- Serge Knystautas <sergek@lokitech.com>
> wrote:
> > > 
> > > > Gabor Kincses wrote:
> > > > > Ok, I found it. 
> MimeMessageInputStreamSource
> > > > should
> > > > > really be called
> > > MimeMessageFileInputStreamSource,
> > > > > then a MimeMessageByteArrayInputStreamSource
> > > could
> > > > be
> > > > > written, which stores the freshly parsed out
> > > > message
> > > > > in  a static hash of ByteArrayInputStreams
> > > instead
> > > > of
> > > > > temp files.  The dispose method would remove
> > the
> > > > entry
> > > > > from the hash.
> > > > 
> > > > Sounds good, though I'd recommend using
> commons
> > > io's
> > > > 
> > > > bytearrayoutputstream instead of the one with
> > the
> > > > JDK.
> > > > 
> > > > -- 
> > > > Serge Knystautas
> > > > Lokitech >> software . strategy . design >>
> > > > http://www.lokitech.com
> > > > p. 301.656.5501
> > > > e. sergek@lokitech.com
> > > > 
> > > >
> > >
> >
>
---------------------------------------------------------------------
> > > > To unsubscribe, e-mail:
> > > > server-dev-unsubscribe@james.apache.org
> > > > For additional commands, e-mail:
> > > > server-dev-help@james.apache.org
> > > > 
> > > > 
> > > 
> > > 
> > > =====
> > > Gabor Kincses
> > > Running Mandrake Linux 10.0
> > > 
> > > 
> > > 		
> > > __________________________________ 
> > > Do you Yahoo!? 
> > > Yahoo! Mail - now with 250MB free storage. Learn
> > > more.
> > > http://info.mail.yahoo.com/mail_250
> > > 
> > >
> >
>
---------------------------------------------------------------------
> > > To unsubscribe, e-mail:
> > > server-dev-unsubscribe@james.apache.org
> > > For additional commands, e-mail:
> > > server-dev-help@james.apache.org
> > > 
> > > 
> > 
> > =====
> > Gabor Kincses
> > Running Mandrake Linux 10.0
> > 
> > 
> > 	
> > 		
> > __________________________________ 
> > Do you Yahoo!? 
> > Yahoo! Mail - You care about security. So do we. 
> > http://promotions.yahoo.com/new_mail>
>
/***********************************************************************
> >  * Copyright (c) 2000-2004 The Apache Software
> > Foundation.             *
> >  * All rights reserved.                           
>  
> >                   *
> >  *
> >
>
-------------------------------------------------------------------
> > *
> >  * Licensed under the Apache License, Version 2.0
> > (the "License"); you *
> >  * may not use this file except in compliance with
> > the License. You    *
> >  * may obtain a copy of the License at:           
>  
> >                   *
> >  *                                                
>  
> >                   *
> >  *     http://www.apache.org/licenses/LICENSE-2.0 
>  
> >                   *
> >  *                                                
>  
> >                   *
> >  * Unless required by applicable law or agreed to
> in
> > writing, software *
> >  * distributed under the License is distributed on
> > an "AS IS" BASIS,   *
> >  * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND,
> > either express or     *
> >  * implied.  See the License for the specific
> > language governing       *
> >  * permissions and limitations under the License. 
>  
> >                   *
> > 
> >
>
***********************************************************************/
> > 
> > package org.apache.james.core;
> > 
> > import java.io.*;
> > 
> > import javax.mail.MessagingException;
> > 
> > import
> > org.apache.avalon.framework.activity.Disposable;
> > 
> > /**
> >  * Takes an input stream and creates a repeatable
> > input stream source
> >  * for a MimeMessageWrapper.  It does this by
> > completely reading the
> >  * input stream and saving that to a temporary
> file
> > that should delete on exit,
> >  * or when this object is GC'd.
> >  *
> >  * @see MimeMessageWrapper
> >  *
> >  *
> >  */
> > public class MimeMessageByteArrayInputStreamSource
> >     extends MimeMessageSource
> >     implements Disposable {
> >     
> >     /**
> >      * Random key into the cache
> >      */
> >     String sourceId = null;
> > 
> >     /**
> >      * The mime bytes
> >      */
> >     byte[] bytes = null;
> >     
> >     /**
> >      * @param key
> >      * @param in
> >      * @throws MessagingException
> >      */
> >     /**
> >      * Construct a new
> > MimeMessageByteArrayInputStreamSource from an
> >      * <code>InputStream</code> that contains the
> 
=== message truncated ===


=====
Gabor Kincses
Running Mandrake Linux 10.0


		
__________________________________ 
Do you Yahoo!? 
Yahoo! Mail - now with 250MB free storage. Learn more.
http://info.mail.yahoo.com/mail_250

---------------------------------------------------------------------
To unsubscribe, e-mail: server-dev-unsubscribe@james.apache.org
For additional commands, e-mail: server-dev-help@james.apache.org


Mime
View raw message