mina-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Tomasz Blachowicz" <tblachow...@gmail.com>
Subject Re: Google Protocol Buffer codec...
Date Thu, 08 Jan 2009 17:25:24 GMT
On Thu, Jan 8, 2009 at 6:16 AM, Ashish <paliwalashish@gmail.com> wrote:

> Some observation
> 1. the sandbox folder has whole of MINA code. Will be great if we can
> remove other files


The filter codec I've develop to handle Google Protocol Buffers essentially
lives in his own module mina-filter-codec-protobuf and depends only on
mina-core module. So, if you want to focus only on the code I've
contributed, please consider mina-filter-codec-protobuf  module.


> AFAIK, protocol buffer codec is a binary data binding framework,
> something similar to XML-Data binding framework's like JAXB, Castor,
> XMLBeans. Google's implementation reminds of CORBA IDL's
>
> There is basic operational similarity of Google Protocol Buffer codec
> with prefixed string decoder. We have a length and then actual
> messages. It will be worthwhile to explore if we can align the google
> codec with prefixed string codec.
>

Yes, the prefixed string codec is similar to the one I've developed for
protobuf messages. However there are two things to be considered before we
would start any alignment:
1. Prefixed string codec has been implemented in mina-core module and
protobuf codec resides in it's own *optional* module.
2. Prefixed string codec uses fixed format to encode length of the message
as protobuf codec uses varint way to encode the integer values (
http://code.google.com/apis/protocolbuffers/docs/encoding.html)<http://code.google.com/apis/protocolbuffers/docs/encoding.html>.
This way of encoding is pretty nifty and can save bandwidth if you are
sending many lightweight messages. For instance value less than 128 requires
only two bytes to encode and values less than 16K requires 3. What is more,
prefixed string codec uses method on IoBuffer implementation to determine
the size, while protobuf codec determines the size internally using varints
encoding.

There is one more situation that must be though about is piggybacked
> messages. the implementation can receive a packet with one message
> ending and another message starting (again by nature of TCP). So
> returning true from decoder may result in an exception (more data
> remaining) [Please correct me if I am wrong]


This is not a problem as when the decoder receives a buffer of say 64 bytes
and reads only 50 of them to decode the message (and returns true and then
the framework will present remaining 14 bytes in the next call to the decode
method. Please refer to the test cases I've developed.


> Also, is it worthwhile to have a mechanism to add extensions into
> ExtensionRegistry? or the library is doing it.


I spent some time playing around with Google Protocol Buffers and in my
opinion the extensions is one of the strongest features of the library.
Because of the fact that the protobuf is not self-descriptive protocol
(as opposite to Java serialization) the meta data about the message carried
over the wire is very limited and have to be provided to the decoder by
the application that consumes the messages. However I'm open to all the
advices and suggestions how we might do it better :)


> Would like to spend some more time with the code before commenting
> more on this. Though it will be worthwhile to have this codec with us.
> Once we review the code, lets vote for the same.


Any comments are welcome and highly appreciated!


> Shall try to run this implementation and validate it against my
> possible use case. Though would have loved to have an implementation
> where I don't have to define .proto files. (I know the implementation
> doesn't work that ways)


Well, in my opinion it'd be madness to write the Java class for the message
by hand. What I can do is I can provide you with the maven code snippet that
will allow you automatically generate Java code from .proto files while you
build the application. How does it sound?

Cheers,
Tom

On Thu, Jan 8, 2009 at 2:50 AM, Emmanuel Lecharny <elecharny@gmail.com>
> wrote:
> > Hi guys,
> >
> > a couple of weeks ago, Thomasz Blachowicz submitted a patch including
> some
> > Google protocol buffer codec (see
> > https://issues.apache.org/jira/browse/DIRMINA-654).
> >
> > I just injected it into the sandbox, for us to review the code :
> > http://svn.apache.org/viewvc?rev=732497&view=rev
> >
> > If some of us can review the code and tell if it's ok, we then would be
> able
> > to inject it in the project (maybe after 2.0, depends).
> >
> > Or if you have any suggestion, or think that it's not worth being a part
> of
> > MINA, feel free to give some feedback.
>

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message