kafka-users mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Neha Narkhede <neha.narkh...@gmail.com>
Subject Re: corrupted topic
Date Tue, 25 Oct 2011 17:17:28 GMT
Tim,

Thanks for looking into this !

>> it looked as if some of them use the same logic as the ruby client, and might also
be affected.

Could you please list the clients that you think might have the same bug ?

> are you actually using any client besides the java / scala one in production at linkedin?

No.

Thanks,
Neha

On Tue, Oct 25, 2011 at 9:16 AM, Tim Lossen <tim@lossen.de> wrote:
> jun,
>
> the ruby client (maintained by alejandro crosa) is here:
>
> https://github.com/acrosa/kafka-rb
>
> i noticed just now that he also seems to work at linkedin?
>
> last commit on github is a bugfix, on october 14. but there
> also seem to be some changes on the apache side which are not
> in the github version ... maybe alejandro can best sort this
> out himself.
>
> while looking over the other clients when we where hunting for
> this bug, it looked as if some of them use the same logic as
> the ruby client, and might also be affected.
>
> are you actually using any client besides the java / scala one
> in production at linkedin?
>
> cheers
> tim
>
>
> On 2011-10-25, at 17:58 , Jun Rao wrote:
>
>> Tim,
>>
>> Thanks for the update. What's the github url for the ruby client? Has it
>> diverged from what's in Apache?
>>
>> I agree with you that we should consider excluding those clients not well
>> maintained from our distribution.
>>
>> Jun
>>
>> On Tue, Oct 25, 2011 at 7:46 AM, Tim Lossen <tim@lossen.de> wrote:
>>
>>> ok, we finally traced this issue to a bug in the ruby kafka client,
>>> which we were able to fix -- the topic was never corrupted.
>>>
>>> (we sent a pull request to the maintainer of the client on github.)
>>>
>>> BTW, i do not think that it is a good idea to include an (outdated)
>>> copy of the ruby client (and other clients) in the kafka distribution.
>>> maybe better *link* to the actual client projects?
>>>
>>> cheers
>>> tim
>>>
>>> On 2011-10-24, at 21:30 , Tim Lossen wrote:
>>>
>>>> ok, thanks -- tomorrow we'll try investigate further ...
>>>>
>>>>
>>>> On 2011-10-24, at 9:12 PM, Neha Narkhede wrote:
>>>>
>>>>> Tim,
>>>>>
>>>>>> what if the CRC32 checksum is correct, but the internal binary message
>>> structure is not?
>>>>>
>>>>> The CRC check involves recomputing the CRC and then checking against
>>>>> the stored CRC in the header. The probability of that matching is
>>>>> extremely low.
>>>>>
>>>>> Corruption is also possible if the broker crashes in the middle of a
>>>>> flush. In that case, when the broker restarts, it detects an unclean
>>>>> shutdown, runs recovery on the logs and truncates the log if the CRC
>>>>> check fails at some message.
>>>>>
>>>>> Also, we compute the CRC only on the payload of the message. So
>>>>> technically, some bits could get flipped in the header of the message.
>>>>>
>>>>> Thanks,
>>>>> Neha
>>>>>
>>>>> On Mon, Oct 24, 2011 at 12:07 PM, Tim Lossen <tim@lossen.de> wrote:
>>>>>> what if the CRC32 checksum is correct, but the internal binary message
>>>>>> structure is not?
>>>>>>
>>>>>>
>>>>>> On 2011-10-24, at 8:56 PM, Jay Kreps wrote:
>>>>>>
>>>>>>> It is not supposed to be possible. We include a CRC32 with each
>>> message,
>>>>>>> so
>>>>>>> invalid requests should be detected and rejected. But that does
not
>>>>>>> preclude
>>>>>>> the possibility that we missed a case.
>>>>>>>
>>>>>>> -Jay
>>>>>>>
>>>>>>> On Mon, Oct 24, 2011 at 11:41 AM, Tim Lossen <tim@lossen.de>
wrote:
>>>>>>>
>>>>>>>> hi,
>>>>>>>>
>>>>>>>> is it possible for a faulty client to "corrupt" a topic on
the
>>> broker,
>>>>>>>> so that consumers cannot consume messages any more? or does
>>>>>>>> the broker protect itself against this?
>>>>>>>>
>>>>>>>> i am asking because we seem to have run into such a situation.
>>>>>>>> we are using a perl producer and a ruby consumer. the per
lib might
>>>>>>>> be a bit outdated.
>>>>>>>>
>>>>>>>> cheers
>>>>>>>> tim
>>>>>>>>
>>>>>>>> --
>>>>>>>> http://tim.lossen.de
>>>>>>>>
>>>>>>>>
>>>>>>
>>>>>> --
>>>>>> http://tim.lossen.de
>>>>>>
>>>>>>
>>>>
>>>> --
>>>> http://tim.lossen.de
>>>>
>>>
>>> --
>>> http://tim.lossen.de
>>>
>>>
>>>
>>>
>
> --
> http://tim.lossen.de
>
>
>
>

Mime
View raw message