james-server-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Norman Maurer <norman.mau...@googlemail.com>
Subject Re: HBase message implementation
Date Mon, 01 Aug 2011 16:52:30 GMT
Hi Eugen,

comments inside..

2011/8/1 Ioan Eugen Stan <stan.ieugen@gmail.com>:
> Hello,
>
> I'm having some issues putting things together when it comes to
> Message implementation. I will explain things right away.
> The current code [1], [2], and [3] is based on JPA implementation and
> it copies the message content into byte arrays.
> This does not seem to scale too well.
>
> There is also the issue of getting the info back from HBase. I agree
> with Eric and Norman that large content should be split and I'm
> thinking of providing a HBaseMessage implementation that should read
> data from HBase as demanded (ChunkedInputStream and
> ChunkedOutputStream as suggested by Norman [4] ).
>
> Do I have to copy the data when someone creates a new HBaseMessage, or
> as suggested by the streaming alternative, I can save a reference of
> SharedInputStream and when I save the message to the mailbox I can
> move the bytes to HBase.
>
> The way I see things is that HBaseMessage implementation is only used
> for retrieving data from HBase. It should not store the message body
> (in the constructor).

Exactly.. The only special case is the MessageMapper.copy(..) but
maybe thiscan be handled in a more efficient way in hbase...

>
> Please, I need some clarification.
>
> Thanks,
>
> [1] http://code.google.com/a/apache-extras.org/p/mailbox-hdfs/source/browse/src/main/java/org/apache/james/mailbox/hbase/mail/model/hbase/HBaseMessage.java
> [2] http://code.google.com/a/apache-extras.org/p/mailbox-hdfs/source/browse/src/main/java/org/apache/james/mailbox/hbase/mail/model/hbase/AbstractHBaseMessage.java
> [3] http://code.google.com/a/apache-extras.org/p/mailbox-hdfs/source/browse/src/main/java/org/apache/james/mailbox/hbase/mail/model/hbase/HBaseStreamingMessage.java
> [4] https://github.com/rantav/hector/tree/master/core/src/main/java/me/prettyprint/cassandra/io
>
> --
> Ioan Eugen Stan
> http://ieugen.blogspot.com/


Bye,
Norman

---------------------------------------------------------------------
To unsubscribe, e-mail: server-dev-unsubscribe@james.apache.org
For additional commands, e-mail: server-dev-help@james.apache.org


Mime
View raw message