kafka-users mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Jay Kreps <jay.kr...@gmail.com>
Subject Re: 0.8 producer -- many questions
Date Fri, 07 Dec 2012 17:15:59 GMT
I agree that BNFs are not the most readable thing in the world, but the
fact is that our protocol has variable length fields and lots of repeated
structures which are pretty confusing in a positional form. In practice I
think the code one writes looks a lot like the BNF (e.g.
readTopicMetadataRequest() calla readRequestHeader() and
readArray(readString)). What does a more complex structure like the
TopicMetadataResponse look like in your style of documentation?

-Jay


On Fri, Dec 7, 2012 at 5:40 AM, ben fleis <ben@monkey.org> wrote:

> First off, let me amend my previous votes: +1 for both ApiKey and
> ApiVersion in all responses, even @ 4 bytes.  The number of bytes is
> insignificant, useful in the debugging and multiplex scenarios, even if the
> underlying service doesn't yet support it.  (The protocol still can.)
>  CorrelationId can clearly be used to the same end, but it's still my
> preference.
>
> Secondly, things changed since yesterday, so now i'm using the doc to
> update my protocol.
>
> Biggest comment -- the visual layout of the wire protocol doesn't reflect
> the way I want to read a protocol.  (IMHO, BNF makes sense for token/string
> based protocols, but not so much for binary.)  Alignment, byte counts when
> available, etc. would all be fantastic.  This comes from my own notes, and
> gives an idea of what I wanted to know while implementing:
>
> StandardRequestHeader:
>
>
>     field name                | type              | bytes   | notes
>
>
> ----------------------------------------------------------------------------
>     size                      | int32             | 4       | remaining
> packet size
>     api_key                   | int16             | 2       | currently
> [0,5]
>     api_version               | int16             | 2       |
>     correlation_id            | int32             | 4       |
>     client_id                 | int16_string      | 2+|s|   |
>
> TopicMetadata Request:
>
>     field name                | type              | bytes   | notes
>
>
> ----------------------------------------------------------------------------
>     <StandardRequestHeader>                       | 14+|s|  |
>     len([topic name])         | int32             | 4       |
>     [topic name]              | [int16_string]    | [2+|s|] |
>
> I will have more commentary as I read more, but this is the major thing for
> me.
>
> b
>

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message