james-server-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "stack (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (MAILBOX-44) [gsoc2011] Design and implement a distributed mailbox using Hadoop
Date Tue, 14 Jun 2011 22:48:47 GMT

    [ https://issues.apache.org/jira/browse/MAILBOX-44?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13049492#comment-13049492
] 

stack commented on MAILBOX-44:
------------------------------

All mail in a single row in hbase would mean that the mailbox would be changed 'atomically'
since row updates in hbase are so but downsides might be that that some users would have really
big mailboxes and gigabyte-sized rows; this might mess w/ balance and distribution of across
the cluster (perhaps).

If you did put them all in a single row, in hbase columns are sorted too; if the column qualifier
were a reverse order date you could encounter mail in order of newest first.  HBase has versioning
too so you could stamp mail into hbase and write the mail receipt date as the cell version.
 Naturally it returns versions in order of newest first.

How would you do threading?  Does James support this?  What else does James support that you
expect the db to provide?

> [gsoc2011] Design and implement a distributed mailbox using Hadoop
> ------------------------------------------------------------------
>
>                 Key: MAILBOX-44
>                 URL: https://issues.apache.org/jira/browse/MAILBOX-44
>             Project: James Mailbox
>          Issue Type: New Feature
>            Reporter: Eric Charles
>            Assignee: Norman Maurer
>              Labels: gsoc2011
>             Fix For: 0.3
>
>
> Context: The mailbox subproject (http://james.apache.org/mailbox/) supports maildir,
SQL database (via JPA) and Java Content Repository (JCR) as technology for mail storage. This
flexibility is achieved thanks to a API design that abstracts mail storage from the mail protocols.
> Task: We need to implement mailbox storage as a distributed system on top of Hadoop HDFS.
The James mailbox API will be used. A first step is to design how to interact with Hadoop
(native api, gora incubator at apache,...) and deal with specific performance questions related
to mail loading/parsing in a distributed system (use map/reduce or not, use existing local
lucene indexes for search,...). The second step is to implement the HDFS mailbox (maildir
mailbox is similar because is stores mails as a file and can be an inspiration). A single
James server will still be deployed because we don't have any distributed UID generation.
> Mentor: eric at apache dot org
> Complexity: medium 

--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira

        

---------------------------------------------------------------------
To unsubscribe, e-mail: server-dev-unsubscribe@james.apache.org
For additional commands, e-mail: server-dev-help@james.apache.org


Mime
View raw message