kafka-users mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Dana Powers <dana.pow...@gmail.com>
Subject Re: Protocol Question
Date Thu, 04 Feb 2016 04:32:45 GMT
Hi Heath, a few comments:

(1) you should be looping on brokerCount
(2) you are missing the topic array count (4 bytes), and loop
(3) topic error code is int16, so only 2 bytes not 4
(4) you are missing the partition metadata array count (4 bytes), and loop
(5) you are missing the replicas and isr parsing.

-Dana

On Wed, Feb 3, 2016 at 6:01 PM, Heath Ivie <hivie@autoanything.com> wrote:

> Hi,
>
> I am trying to navigate through the protocol and I am seeing some
> inconsistencies with the data.
>
> I am trying to parse out the MetadataResponse and I am seeing bytes in
> between where they shouldn't be.
>
> I know they are extra, because if I up the offset the data after is
> correct.
>
> Here is the part of the response that I am trying to parse:
> MetadataResponse => [Broker][TopicMetadata]
>   Broker => NodeId Host Port  (any number of brokers may be returned)
>     NodeId => int32
>     Host => string
>     Port => int32
>   TopicMetadata => TopicErrorCode TopicName [PartitionMetadata]
>     TopicErrorCode => int16
>   PartitionMetadata => PartitionErrorCode PartitionId Leader Replicas Isr
>     PartitionErrorCode => int16
>     PartitionId => int32
>     Leader => int32
>     Replicas => [int32]
>     Isr => [int32]
>
>
> I am seeing extra bytes after the topic error.
>
> I am not sure if this padding is expected or not?
>
> It may not be very readable, but here is my code (be nice :)).
>
> int offset =0;
>                     int correlationId =
> BitConverter.ToInt32(ReverseBytes(bytes.Skip(offset).Take(4).ToArray()), 0);
>                     offset += 4;
>                     int brokerCount =
> BitConverter.ToInt32(ReverseBytes(bytes.Skip(offset).Take(4).ToArray()), 0);
>                     offset += 4;
>                     int nodeId =
> BitConverter.ToInt32(ReverseBytes(bytes.Skip(offset).Take(4).ToArray()), 0);
>                     offset += 4;
>                     int hostNameLength =
> BitConverter.ToInt16(ReverseBytes(bytes.Skip(offset).Take(2).ToArray()), 0);
>                     offset += 2;
>
>                     string hostName = Encoding.ASCII.GetString(bytes,
> offset, hostNameLength);
>                     offset += hostNameLength;
>
>                     int port =
> BitConverter.ToInt32(ReverseBytes(bytes.Skip(offset).Take(4).ToArray()), 0);
>                     offset += 4;
>
>                     int topicErrorCode =
> BitConverter.ToInt32(ReverseBytes(bytes.Skip(offset).Take(4).ToArray()), 0);
>                     offset += 4;
>
>                     int topicNameLength =
> BitConverter.ToInt16(ReverseBytes(bytes.Skip(offset).Take(2).ToArray()), 0);
>                     offset += 2;
>
>                     string topicName = Encoding.ASCII.GetString(bytes,
> offset, topicNameLength);
>                     offset += topicNameLength;
>
>                     int partitionErrorCode =
> BitConverter.ToInt16(ReverseBytes(bytes.Skip(offset).Take(2).ToArray()), 0);
>                     offset += 2;
>
>                     int partitionId =
> BitConverter.ToInt32(ReverseBytes(bytes.Skip(offset).Take(4).ToArray()), 0);
>                     offset += 4;
>
>                     int leaderId =
> BitConverter.ToInt32(ReverseBytes(bytes.Skip(offset).Take(4).ToArray()), 0);
>                     offset += 4;
>
>
> Any help would be cool, thanks
>
> Heath Ivie
> Solutions Architect
>
>
> Warning: This e-mail may contain information proprietary to AutoAnything
> Inc. and is intended only for the use of the intended recipient(s). If the
> reader of this message is not the intended recipient(s), you have received
> this message in error and any review, dissemination, distribution or
> copying of this message is strictly prohibited. If you have received this
> message in error, please notify the sender immediately and delete all
> copies.
>

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message