storm-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Alexandre Vermeerbergen <avermeerber...@gmail.com>
Subject Re: Is it possible to choose Kafa version with StormKafka from Storm 1.0.2 ?
Date Thu, 12 Jan 2017 09:19:34 GMT
Hello Jungtae,

Thank you very much for your reply.
Is it possible for us to used storm-kafka-client 1.1.0-SNAPSHOT in our fat
jar, but deploy this fat jar with our existing Storm 1.0.1 cluster ; or do
we need to also upgrade our cluster to 1.1.0-SNAPSHOT as well (I hope not) ?

Best regards,
Alexandre


2017-01-12 9:48 GMT+01:00 Jungtaek Lim <kabhwan@gmail.com>:

> Hi Alexandre,
>
> README of storm-kafka describes how to change Kafka version.
> https://github.com/apache/storm/tree/master/external/
> storm-kafka#using-storm-kafka-with-different-versions-of-kafka
>
> As README stated, if you use Kafka 0.10.x, please try out new module
> storm-kafka-client which is using new client API.
>
> storm-kafka-client 1.0.2 is compatible with Kafka 0.9, and
> storm-kafka-client 1.1.0 SNAPSHOT is compatible with Kafka 0.10.x.
>
> I think we're close to start release process of Storm 1.1.0. Before that
> you may need to build Storm 1.x branch via mvn install, and include
> storm-kafka-client 1.1.0-SNAPSHOT as your fat jar.
>
> Thanks,
> Jungtaek Lim
>
> 2017년 1월 12일 (목) 오후 12:30, Aaron Niskodé-Dossett <dossett@gmail.com>님이
작성:
>
> Hi Alexandre,
>
> My recollection from trying this when 0.10 was released is that it is
> possible, but some small code changes are necessary.  If you're comfortable
> building Storm from source, updating the kafka version in the appropriate
> poms and compiling would show you what needs to be updated.
>
> On Wed, Jan 11, 2017 at 2:20 PM Alexandre Vermeerbergen <
> avermeerbergen@gmail.com> wrote:
>
> Hello,
>
> Any clue on whether or not it's possible to build a KafkaSpout / KafkaBolt
> based on Apache Kafka 0.10.x ?
>
> Best regards,
> Alexandre
>
>
> 2017-01-07 18:03 GMT+01:00 Alexandre Vermeerbergen <
> avermeerbergen@gmail.com>:
>
> Hello,
>
> We have been happyly running our real-time supervision homebrewed
> application with Apache Storm 1.0.1 and its out-of-the-box StormKafka spout
> & bold with our Kafka Broker 0.9.0.1 cluster for more than 6 months.
>
> We're been recently interested in upgrading our setup to Apache Storm
> 1.0.2, mainly because we sometime hit https://issues.apache.org/
> jira/browse/STORM-1879
>
> We also plan to soon upgrade our Kafka Broker cluster to 0.10.x
>
> However, we noticed that Storm 1.0.2's KafkaSpout has been downgraded to
> Kafka 0.8.2.2 : STORM-1755: Revert the kafka client version upgrade in
> storm-kafka module
>
> Kafka 0.10 brokers have a new message format which can be consumed and
> produced to using older Kafka API, but at the price of performance
>
> Is it posible to build a KafkaStorm support based on Kafka 0.10.x API ?
>
> Best regards,
> Alexandre
>
>
>

Mime
View raw message