kafka-users mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Richard Lee <rd...@tivo.com>
Subject Re: are 0.8.2.1 and 0.9.0.0 compatible?
Date Thu, 01 Oct 2015 20:49:36 GMT
Great.. that makes sense.  Forward compatibility by brokers is likely 
hard, tho it would be nice if clients were backward compatible.  I 
guess, tho, implementing that requires KIP-35.

Thanks for the 0.9.0.0 rolling update pointer.

Richard

On 10/01/2015 10:48 AM, Grant Henke wrote:
> Hi Richard,
>
> You are correct that version will now be 0.9.0 and anything referencing
> 0.8.3 is being changed. You are also correct in the there have been wire
> protocol changes that break compatibility. However, backwards compatibility
> exists and you should always upgrade your brokers before upgrading your
> clients in order to avoid issues (In the future KIP-35
> <http://KIP-35+-+Retrieving+protocol+version> may change that).
>
> It's also worth noting that if you are performing a rolling upgrade of your
> brokers, you need to be sure brokers running the new protocol know to
> communicate with the old version to remain compatible during the bounce.
> This is done using the inter.broker.protocol.version property. More on that
> topic can be read here:
> https://kafka.apache.org/083/documentation.html#upgrade
>
> Hopefully that helps clear things up.
>
> Thank you,
> Grant
>
>
>
>
>
> On Thu, Oct 1, 2015 at 12:21 PM, Richard Lee <rdlee@tivo.com> wrote:
>
>> Note the 0.8.3-SNAPSHOT has recently been renamed 0.9.0.0-SNAPSHOT.
>>
>> In any event, the major version number change could indicate that there
>> has, in fact, been some sort of incompatible change.  Using 0.9.0.0, I'm
>> also unable to use the kafka-console-consumer.sh to read from a 0.8.2.1
>> broker, but it works fine with a 0.9.0.0 broker.
>>
>> Some validation from a kafka expert that broker forward compatibility (or
>> client backward compatibility) is not supported would be appreciated, and
>> that this isn't just a case of some sort of local, fixable misconfiguration.
>>
>> Thanks!
>> Richard
>>
>> On 09/30/2015 11:17 AM, Doug Tomm wrote:
>>
>>> hello,
>>>
>>> i've got a set of broker nodes running 0.8.2.1.  on my laptop i'm also
>>> running 0.8.2.1, and i have a single broker node and mirrormaker there.
>>> i'm also using kafka-console-consumer.sh on the mac to display messages on
>>> a favorite topic being published from the broker nodes.  there are no
>>> messages on the topic, but everything is well-behaved.  i can inject
>>> messages with kafkacat and everything is fine.
>>>
>>> but then!
>>>
>>> on the laptop i switched everything to 0.8.3 but left the broker nodes
>>> alone.  now when i run mirrormaker i see this:
>>>
>>> [2015-09-30 10:44:55,090] WARN
>>> [ConsumerFetcherThread-tivo_kafka_110339-mbpr.local-1443635093396-c55cbafb-0-5],
>>> Error in fetch kafka.consumer.ConsumerFetcherThread$FetchRequest@61cb11c5.
>>> Possible cause: java.nio.BufferUnderflowException
>>> (kafka.consumer.ConsumerFetcherThread)
>>> [2015-09-30 10:44:55,624] WARN
>>> [ConsumerFetcherThread-tivo_kafka_110339-mbpr.local-1443635093396-c55cbafb-0-5],
>>> Error in fetch kafka.consumer.ConsumerFetcherThread$FetchRequest@3c7bb986.
>>> Possible cause: java.nio.BufferUnderflowException
>>> (kafka.consumer.ConsumerFetcherThread)
>>> [2015-09-30 10:44:56,181] WARN
>>> [ConsumerFetcherThread-tivo_kafka_110339-mbpr.local-1443635093396-c55cbafb-0-5],
>>> Error in fetch kafka.consumer.ConsumerFetcherThread$FetchRequest@1d4fbd2c.
>>> Possible cause: java.nio.BufferUnderflowException
>>> (kafka.consumer.ConsumerFetcherThread)
>>> [2015-09-30 10:44:56,726] WARN
>>> [ConsumerFetcherThread-tivo_kafka_110339-mbpr.local-1443635093396-c55cbafb-0-5],
>>> Error in fetch kafka.consumer.ConsumerFetcherThread$FetchRequest@59e67b2f.
>>> Possible cause: java.nio.BufferUnderflowException
>>> (kafka.consumer.ConsumerFetcherThread)
>>>
>>> if i use kafkacat to generate a message on the topic i see
>>> IllegalArgumentExceptions instead.
>>>
>>> this suggests that the two versions of kafka aren't compatible. is this
>>> the case?  does the whole ecosystem need to be on the same version?
>>>
>>> thank you,
>>> doug
>>>
>>>
>


Mime
View raw message