james-server-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Serge Knystautas <ser...@lokitech.com>
Subject Re: Stream-based MIME Parser
Date Fri, 12 Dec 2003 03:40:17 GMT
Noel J. Bergman wrote:
> We have to get away from loading entire messages into the heap unless we are
> editing them.
> What I really could use is a good stream-based MIME parser.  I am starting
> to look at the W3C code and grendel code.  The latter appears to be
> JavaMail-ish, but the W3C code uses callbacks and streams, and looks like a
> more promising approach.
> Any other ideas?

Not sure about what other code there is. :(

I think we frankly need performance efficient editing as well.  While 
avoiding parsing everything is nice, I think key would allowing you to 
parsing and modifying individual parts without touching others. 
Otherwise the optimizations break down when you have to add footer text 
or need to check something in the text message without parsing and 
loading the 5 meg attachment.

It's not insurmountable since mime messages separate rather cleanly.  If 
you can allow it to parse and modifying individual mime parts distictly 
at a time, letting nesting happening as additional stages as well, then 
you could do ok.

Also, I would bet memory allocation is the bigger CPU drain than 
parsing.  Mime messages are mainly just scanning for a long byte 
signature for breaks, maybe sometimes base64 decoding.  But loading a 5 
meg message into a byte[] can take forever with all the reallocations. 
Probably sending stuff to temp disk files as you parse it would help a lot.

Serge Knystautas
Lokitech >> software . strategy . design >> http://www.lokitech.com
p. 301.656.5501
e. sergek@lokitech.com

To unsubscribe, e-mail: server-dev-unsubscribe@james.apache.org
For additional commands, e-mail: server-dev-help@james.apache.org

View raw message