james-server-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Eric Charles <eric.char...@u-mangate.com>
Subject Re: JPAStressTest with derby, h2 and mysql
Date Sun, 02 May 2010 08:37:23 GMT
Hi Norman,

Trying to simulate a more real-life test case, I developed a small 
bomber class that launches parallel threads. Each thread sends mails 
with attachements (with small pause between each mail sending).

I tested with embedded derby with JDBCDomainList.

Depending on the parameters (10 to 100 parallel threads, 5 ms to 100 ms 
pause, 1KB to 100KB attachement), I could have on my dev laptop 5 to 10 
mails per second being processed.

The good think is that I have no exception at all during stable phase.

However, I was not able to have more mails being processed on my latptop 
due to disk access. The CPU and memory are really not charged (20 - 30 
%), but disk is really working non-stop.
This is probably the price to pay for activemq and the kahadb.

Complete scenario with unstable phase:
- Bomb 5 minutes
- Stop bomb.
- Still some mails are spooled for a while which is quite normal.
- When all mails are spooled, disk still runs crazy for minutes, console 
showing a few messages "Slow KahaDB access: cleanup took 684"
- rebombing : org.apache.camel.CamelExecutionException: Exception 
occurred during execution on the exchange: Exchange[Message: 
org.apache.james.core.MailImpl@16863f2] Caused by: java.io.IOException: 
Too many open files - error message in the bomber
- Server console show still a few mails being spooled.
- After, server does not respond on socket - no more exception.

Disk congestion may come from my disk.
I will setup a server environment with good disk, and will rerun the 
test in a few days.
A database on a separate server could also help.

Btw, what should we consider as acceptable/needed as load (xxx mails 
from yyy parallel clients) ?

Tks,

Eric



On 05/01/2010 09:25 AM, Norman Maurer wrote:
> About 6000, then I run out of heap space..
>
> Can you maybe try to deploy it with james and bomb it via smtp ? I was
> not able to see any exception.
>
> Thx,
> Norman
>
>
> 2010/5/1 Eric Charles<eric.charles@u-mangate.com>:
>    
>> Hi Norman,
>>
>> Similar exceptions with properties.put("openjpa.LockTimeout", "30000");
>>
>> I think the environment simply can not follow the load (more than one mail
>> sent each ms).
>> Bootleneck can be anywhere (connection creation, database lock,
>> openjpa,...).
>> This is why I put a sleep between each mail sending.
>> Real servers will better hold the pressure.
>>
>> So I think I'm limited by my environment : it does not allow me to have a
>> really heavy load to validate that IMAP-137 is resolved.
>> Well, it is probably, as before you commits, I directly had the duplicate
>> key exception.
>>
>> How much mails can you hold on your env with h2 inmemory ?
>>
>> Tks,
>>
>> Eric
>>
>>
>> On 05/01/2010 09:08 AM, Norman Maurer wrote:
>>      
>>> Hi Eric,
>>>
>>> thx for your tests. The order of the uid is caused by the threads,
>>> thats nothing to worry about.
>>>
>>> About the timeout, can you try to add this to the test case:
>>>
>>> properties.put("openjpa.LockTimeout", "30000");
>>>
>>> And keep in mind that we don't use any connection pooling in the test
>>> case so the performance should be better when use it in deployment.
>>>
>>> Bye,
>>> Norman
>>>
>>>
>>> 2010/5/1 Eric Charles<eric.charles@u-mangate.com>:
>>>
>>>        
>>>> Hi,
>>>>
>>>> I just tested the JPAStressTest (for IMAP-137 JPA fails to persist
>>>> MailboxMembership Entity on heavy load) Norman committed yesterday.
>>>>
>>>> I made some tests towards derby (memory and embedded) , h2 (memory and
>>>> embedded) and mysql with following configs (and adding the needed jdbc
>>>> drivers in pom.xml):
>>>>
>>>>         // Derby Memory
>>>>         properties.put("openjpa.ConnectionDriverName",
>>>> org.apache.derby.jdbc.EmbeddedDriver.class.getName());
>>>>         properties.put("openjpa.ConnectionURL",
>>>> "jdbc:derby:memory:derbyimap;create=");
>>>>
>>>>         // Derby Embedded
>>>> //        properties.put("openjpa.ConnectionDriverName",
>>>> org.apache.derby.jdbc.EmbeddedDriver.class.getName());
>>>> //        properties.put("openjpa.ConnectionURL",
>>>> "jdbc:derby:derbyimap;create=;user=root;password=root");
>>>>
>>>>         // H2 Memory
>>>> //        properties.put("openjpa.ConnectionDriverName",
>>>> org.h2.Driver.class.getName());
>>>> //        properties.put("openjpa.ConnectionURL",
>>>> "jdbc:h2:mem:h2imap;DB_CLOSE_DELAY=;
>>>>
>>>>         // H2 Embedded
>>>> //        properties.put("openjpa.ConnectionDriverName",
>>>> org.h2.Driver.class.getName());
>>>> //        properties.put("openjpa.ConnectionURL",
>>>> "jdbc:h2:~/h2/h2imap;USER=;PASSWORD=root");
>>>>
>>>>         //Mysql
>>>> //        properties.put("openjpa.ConnectionDriverName",
>>>> com.mysql.jdbc.Driver.class.getName());
>>>> //        properties.put("openjpa.ConnectionURL",
>>>>
>>>> "jdbc:mysql://localhost:3306/msqlimap?user=&password=root&createDatabaseIfNotExist=true&amp;useUnicode=true&amp;characterEncoding=utf-8");
>>>>
>>>>
>>>> I first changed the maximum number of messages from 1.000 to 100.000 and
>>>> saw
>>>> on my linux dev PC (james server, stresstest and databases all running on
>>>> same dev PC):
>>>>
>>>> Derby: after many mails sending:
>>>> java.io.IOException: Too many open files
>>>> ...
>>>> Exception in thread "pool-1-thread-1044" java.lang.NoClassDefFoundError:
>>>> org/apache/james/imap/api/display/HumanReadableText
>>>> ...
>>>>
>>>> H2: after 400 mails:
>>>> java.io.IOException: Too many open files
>>>> ...
>>>> Caused by:<openjpa-1.2.2-r422266:898935 nonfatal general error>
>>>> org.apache.openjpa.persistence.PersistenceException: Timeout trying to
>>>> lock
>>>> table MEMBERSHIP [50200-79] {prepstmnt 406576
>>>> INSERT INTO Membership (mailboxId, uid, answered, deleted, draft,
>>>>         flagged, internalDate, recent, seen, size, MESSAGE_ID)
>>>>     VALUES (?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?)
>>>> [params=g) 1, (long) 49, (int) 0, (int) 0, (int) 0, (int) 0,
>>>> (Timestamp)
>>>> 2010-05-01 08:37:10.293, (int) 0, (int) 0, (int) 25, (long) 4151]}
>>>> [codeP200, state=T00]
>>>> ...
>>>>
>>>> MySql : after +/- 1000 (always around 1002 and 1007) mails :
>>>> Exception in thread "pool-1-thread-84"<openjpa-1.2.2-r422266:898935
>>>> nonfatal user error>    org.apache.openjpa.persistence.NoResultException:
>>>> The
>>>> query on candidate type "class
>>>> org.apache.james.imap.jpa.mail.model.JPAMailbox" with filter "SELECT
>>>> mailbox
>>>> FROM Mailbox mailbox WHERE mailbox.mailboxId =Param" was configured to
>>>> have a unique result, but no instance matched the query.
>>>> ....
>>>> org.apache.james.imap.mailbox.MailboxNotFoundException: Mailbox
>>>> '#mail.INBOX' not found.
>>>> ...
>>>>
>>>> 3 different databases, 3 different behaviour/exceptions.
>>>>
>>>> Running 3 components (james server, stresstest, database) on my
>>>> "not-so-strong-PC" may be an issue.
>>>>
>>>> To make the tests successful, I added a Thread.sleep(5) (15 ms for
>>>> embedded
>>>> derby and h2) between each mail sending.
>>>>
>>>>
>>>> I suppose we can say that in a real environment, the load will never be
>>>> that
>>>> heavy, and that real server will perform better.
>>>>
>>>>
>>>> I also saw that the uid order is not respected:
>>>>
>>>> Append message with uidb58
>>>> Append message with uidb59
>>>> Append message with uidb56
>>>> Append message with uidb57
>>>>
>>>> This could come from a print mismatch, and not from uid being generated
>>>> in a
>>>> bad order.
>>>>
>>>> Tks in advance for your feedbacks, if any,
>>>>
>>>>
>>>> Eric
>>>>
>>>>
>>>> ---------------------------------------------------------------------
>>>> To unsubscribe, e-mail: server-dev-unsubscribe@james.apache.org
>>>> For additional commands, e-mail: server-dev-help@james.apache.org
>>>>
>>>>
>>>>
>>>>          
>>> ---------------------------------------------------------------------
>>> To unsubscribe, e-mail: server-dev-unsubscribe@james.apache.org
>>> For additional commands, e-mail: server-dev-help@james.apache.org
>>>
>>>
>>>        
>>
>> ---------------------------------------------------------------------
>> To unsubscribe, e-mail: server-dev-unsubscribe@james.apache.org
>> For additional commands, e-mail: server-dev-help@james.apache.org
>>
>>
>>      
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: server-dev-unsubscribe@james.apache.org
> For additional commands, e-mail: server-dev-help@james.apache.org
>
>    


---------------------------------------------------------------------
To unsubscribe, e-mail: server-dev-unsubscribe@james.apache.org
For additional commands, e-mail: server-dev-help@james.apache.org


Mime
View raw message