storm-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Alexandre Vermeerbergen <avermeerber...@gmail.com>
Subject Re: Is it possible to choose Kafa version with StormKafka from Storm 1.0.2 ?
Date Thu, 12 Jan 2017 09:21:27 GMT
Hello Aaron,

Thanks for your reply.
Was your test based on the existing storm-kafka external "lib" or on the
newer "storm-kafka-client" one ?
Do you remember which code changes you had to perform? was that on the
"lib" or on your topology ?

best regards,
Alexandre


2017-01-12 4:30 GMT+01:00 Aaron Niskodé-Dossett <dossett@gmail.com>:

> Hi Alexandre,
>
> My recollection from trying this when 0.10 was released is that it is
> possible, but some small code changes are necessary.  If you're comfortable
> building Storm from source, updating the kafka version in the appropriate
> poms and compiling would show you what needs to be updated.
>
> On Wed, Jan 11, 2017 at 2:20 PM Alexandre Vermeerbergen <
> avermeerbergen@gmail.com> wrote:
>
>> Hello,
>>
>> Any clue on whether or not it's possible to build a KafkaSpout /
>> KafkaBolt based on Apache Kafka 0.10.x ?
>>
>> Best regards,
>> Alexandre
>>
>>
>> 2017-01-07 18:03 GMT+01:00 Alexandre Vermeerbergen <
>> avermeerbergen@gmail.com>:
>>
>> Hello,
>>
>> We have been happyly running our real-time supervision homebrewed
>> application with Apache Storm 1.0.1 and its out-of-the-box StormKafka spout
>> & bold with our Kafka Broker 0.9.0.1 cluster for more than 6 months.
>>
>> We're been recently interested in upgrading our setup to Apache Storm
>> 1.0.2, mainly because we sometime hit https://issues.apache.org/
>> jira/browse/STORM-1879
>>
>> We also plan to soon upgrade our Kafka Broker cluster to 0.10.x
>>
>> However, we noticed that Storm 1.0.2's KafkaSpout has been downgraded to
>> Kafka 0.8.2.2 : STORM-1755: Revert the kafka client version upgrade in
>> storm-kafka module
>>
>> Kafka 0.10 brokers have a new message format which can be consumed and
>> produced to using older Kafka API, but at the price of performance
>>
>> Is it posible to build a KafkaStorm support based on Kafka 0.10.x API ?
>>
>> Best regards,
>> Alexandre
>>
>>
>>

Mime
View raw message