james-server-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Eric Charles <e...@apache.org>
Subject Re: hbase mailbox alpha 2
Date Sun, 21 Aug 2011 08:20:36 GMT
On 20/08/11 20:50, Ioan Eugen Stan wrote:
> Hello,
>
> Things are looking better and better. I have refactored a lot of code, mainly:
> - reduced number of columns to 3 in MESSAGES table. I have plans to
> get them down to 2, but it's ok for now.
> - created Convertor classes for Flags and Properties, as Eric once
> suggested. this also lead to the following benefit:
> - HBaseUtils is much more clean. Another sweep is around the corner.
>
> As it stands now, the test results are:
>
> Tests run: 291, Failures: 0, Errors: 80, Skipped: 0
>

Well done :)

> I get a lot of java.nio.BufferOverflowException and I got some other
> stuff to fix in the flags area, but I think things are going well. One
> thing that kept me was the fact that I was returning mssage UID's in
> the wrong order. The server was expecting them in ascending order and
> because I store them using Long.MAX_VALUE - uid i was getting them in
> descending order. Now it's fixed. I'm using Collections.reverse() to
> switch them.
>

Why about storing them directly in correct order?

> The test runs for about 50 min. I don't know why it takes so much but
> I think it has to do something with the large amount of debugging
> output that's coming out of HBase.
>

Cluster start/stop and table enable on each test takes time.
You  can mitigate it via a junit suite that start hbase cluster once 
(don't forget to fix the maven-surefire-plugin in pom.xml to only 
include **/*SuiteTest.java.

You will have to delete/create the table before each test (or delete 
everything). See also open thread [2] on hbase mailing list to further 
optimize the process.

[1] 
http://svn.apache.org/repos/asf/james/server/trunk/hbase/src/test/java/org/apache/james/JamesServerHBaseSuiteTest.java
[2] http://markmail.org/message/o77ldtza4ojbyoxv

> Stay tuned for more information and check out he latest version in the repo:
>
> http://code.google.com/a/apache-extras.org/p/mailbox-hdfs/
> and:
> http://code.google.com/a/apache-extras.org/p/hbase-mailbox-integration-test/
>
> p.s. a quick code review will be appreciated.
>

I went through the last version, but don't have much time today...
Quick feedback:

- deleteAllMemberships and deleteAllMailboxes: you delete/recreate the 
tables instead of removing the content. why not. you could centralize 
the table creation code somewhere in a lazy singleton.

- HBaseUtils: I would separate the constants definitions in to a new 
interface and leave the methods in the class. Also, you can use inner 
interface to make it still more readable.

- the mm file should not be under src/main/resources, but under 
src/site/resource for example.


> Thanks,

-- 
Eric
http://about.echarles.net

---------------------------------------------------------------------
To unsubscribe, e-mail: server-dev-unsubscribe@james.apache.org
For additional commands, e-mail: server-dev-help@james.apache.org


Mime
View raw message