james-server-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Eric Charles <eric.char...@u-mangate.com>
Subject Re: JPAStressTest with derby, h2 and mysql
Date Sat, 01 May 2010 07:21:25 GMT
Hi Norman,

Similar exceptions with properties.put("openjpa.LockTimeout", "30000");

I think the environment simply can not follow the load (more than one 
mail sent each ms).
Bootleneck can be anywhere (connection creation, database lock, 
openjpa,...).
This is why I put a sleep between each mail sending.
Real servers will better hold the pressure.

So I think I'm limited by my environment : it does not allow me to have 
a really heavy load to validate that IMAP-137 is resolved.
Well, it is probably, as before you commits, I directly had the 
duplicate key exception.

How much mails can you hold on your env with h2 inmemory ?

Tks,

Eric


On 05/01/2010 09:08 AM, Norman Maurer wrote:
> Hi Eric,
>
> thx for your tests. The order of the uid is caused by the threads,
> thats nothing to worry about.
>
> About the timeout, can you try to add this to the test case:
>
> properties.put("openjpa.LockTimeout", "30000");
>
> And keep in mind that we don't use any connection pooling in the test
> case so the performance should be better when use it in deployment.
>
> Bye,
> Norman
>
>
> 2010/5/1 Eric Charles<eric.charles@u-mangate.com>:
>    
>> Hi,
>>
>> I just tested the JPAStressTest (for IMAP-137 JPA fails to persist
>> MailboxMembership Entity on heavy load) Norman committed yesterday.
>>
>> I made some tests towards derby (memory and embedded) , h2 (memory and
>> embedded) and mysql with following configs (and adding the needed jdbc
>> drivers in pom.xml):
>>
>>         // Derby Memory
>>         properties.put("openjpa.ConnectionDriverName",
>> org.apache.derby.jdbc.EmbeddedDriver.class.getName());
>>         properties.put("openjpa.ConnectionURL",
>> "jdbc:derby:memory:derbyimap;create=ue");
>>
>>         // Derby Embedded
>> //        properties.put("openjpa.ConnectionDriverName",
>> org.apache.derby.jdbc.EmbeddedDriver.class.getName());
>> //        properties.put("openjpa.ConnectionURL",
>> "jdbc:derby:derbyimap;create=ue;user=root;password=root");
>>
>>         // H2 Memory
>> //        properties.put("openjpa.ConnectionDriverName",
>> org.h2.Driver.class.getName());
>> //        properties.put("openjpa.ConnectionURL",
>> "jdbc:h2:mem:h2imap;DB_CLOSE_DELAY=");
>>
>>         // H2 Embedded
>> //        properties.put("openjpa.ConnectionDriverName",
>> org.h2.Driver.class.getName());
>> //        properties.put("openjpa.ConnectionURL",
>> "jdbc:h2:~/h2/h2imap;USER=ot;PASSWORD=root");
>>
>>         //Mysql
>> //        properties.put("openjpa.ConnectionDriverName",
>> com.mysql.jdbc.Driver.class.getName());
>> //        properties.put("openjpa.ConnectionURL",
>> "jdbc:mysql://localhost:3306/msqlimap?user=ot&password=root&createDatabaseIfNotExist=true&amp;useUnicode=true&amp;characterEncoding=utf-8");
>>
>>
>> I first changed the maximum number of messages from 1.000 to 100.000 and saw
>> on my linux dev PC (james server, stresstest and databases all running on
>> same dev PC):
>>
>> Derby: after many mails sending:
>> java.io.IOException: Too many open files
>> ...
>> Exception in thread "pool-1-thread-1044" java.lang.NoClassDefFoundError:
>> org/apache/james/imap/api/display/HumanReadableText
>> ...
>>
>> H2: after 400 mails:
>> java.io.IOException: Too many open files
>> ...
>> Caused by:<openjpa-1.2.2-r422266:898935 nonfatal general error>
>> org.apache.openjpa.persistence.PersistenceException: Timeout trying to lock
>> table MEMBERSHIP [50200-79] {prepstmnt 406576
>> INSERT INTO Membership (mailboxId, uid, answered, deleted, draft,
>>         flagged, internalDate, recent, seen, size, MESSAGE_ID)
>>     VALUES (?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?)
>> [params=ong) 1, (long) 49, (int) 0, (int) 0, (int) 0, (int) 0, (Timestamp)
>> 2010-05-01 08:37:10.293, (int) 0, (int) 0, (int) 25, (long) 4151]}
>> [codeP200, state=HYT00]
>> ...
>>
>> MySql : after +/- 1000 (always around 1002 and 1007) mails :
>> Exception in thread "pool-1-thread-84"<openjpa-1.2.2-r422266:898935
>> nonfatal user error>  org.apache.openjpa.persistence.NoResultException: The
>> query on candidate type "class
>> org.apache.james.imap.jpa.mail.model.JPAMailbox" with filter "SELECT mailbox
>> FROM Mailbox mailbox WHERE mailbox.mailboxId =idParam" was configured to
>> have a unique result, but no instance matched the query.
>> ....
>> org.apache.james.imap.mailbox.MailboxNotFoundException: Mailbox
>> '#mail.INBOX' not found.
>> ...
>>
>> 3 different databases, 3 different behaviour/exceptions.
>>
>> Running 3 components (james server, stresstest, database) on my
>> "not-so-strong-PC" may be an issue.
>>
>> To make the tests successful, I added a Thread.sleep(5) (15 ms for embedded
>> derby and h2) between each mail sending.
>>
>>
>> I suppose we can say that in a real environment, the load will never be that
>> heavy, and that real server will perform better.
>>
>>
>> I also saw that the uid order is not respected:
>>
>> Append message with uidb58
>> Append message with uidb59
>> Append message with uidb56
>> Append message with uidb57
>>
>> This could come from a print mismatch, and not from uid being generated in a
>> bad order.
>>
>> Tks in advance for your feedbacks, if any,
>>
>>
>> Eric
>>
>>
>> ---------------------------------------------------------------------
>> To unsubscribe, e-mail: server-dev-unsubscribe@james.apache.org
>> For additional commands, e-mail: server-dev-help@james.apache.org
>>
>>
>>      
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: server-dev-unsubscribe@james.apache.org
> For additional commands, e-mail: server-dev-help@james.apache.org
>
>    


---------------------------------------------------------------------
To unsubscribe, e-mail: server-dev-unsubscribe@james.apache.org
For additional commands, e-mail: server-dev-help@james.apache.org


Mime
View raw message