storm-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Jungtaek Lim <kabh...@gmail.com>
Subject Re: Is it possible to choose Kafa version with StormKafka from Storm 1.0.2 ?
Date Thu, 12 Jan 2017 09:31:24 GMT
Alexandre,

I guess there's no issue on deploying storm-kafka-client 1.1.0-SNAPSHOT to
Storm 1.0.2 cluster since there was no huge change on the API, though I
haven't tried.

- Jungtaek Lim (HeartSaVioR)

2017년 1월 12일 (목) 오후 6:22, Alexandre Vermeerbergen <avermeerbergen@gmail.com>님이
작성:

> Hello Aaron,
>
> Thanks for your reply.
> Was your test based on the existing storm-kafka external "lib" or on the
> newer "storm-kafka-client" one ?
> Do you remember which code changes you had to perform? was that on the
> "lib" or on your topology ?
>
> best regards,
> Alexandre
>
>
> 2017-01-12 4:30 GMT+01:00 Aaron Niskodé-Dossett <dossett@gmail.com>:
>
> Hi Alexandre,
>
> My recollection from trying this when 0.10 was released is that it is
> possible, but some small code changes are necessary.  If you're comfortable
> building Storm from source, updating the kafka version in the appropriate
> poms and compiling would show you what needs to be updated.
>
> On Wed, Jan 11, 2017 at 2:20 PM Alexandre Vermeerbergen <
> avermeerbergen@gmail.com> wrote:
>
> Hello,
>
> Any clue on whether or not it's possible to build a KafkaSpout / KafkaBolt
> based on Apache Kafka 0.10.x ?
>
> Best regards,
> Alexandre
>
>
> 2017-01-07 18:03 GMT+01:00 Alexandre Vermeerbergen <
> avermeerbergen@gmail.com>:
>
> Hello,
>
> We have been happyly running our real-time supervision homebrewed
> application with Apache Storm 1.0.1 and its out-of-the-box StormKafka spout
> & bold with our Kafka Broker 0.9.0.1 cluster for more than 6 months.
>
> We're been recently interested in upgrading our setup to Apache Storm
> 1.0.2, mainly because we sometime hit
> https://issues.apache.org/jira/browse/STORM-1879
>
> We also plan to soon upgrade our Kafka Broker cluster to 0.10.x
>
> However, we noticed that Storm 1.0.2's KafkaSpout has been downgraded to
> Kafka 0.8.2.2 : STORM-1755: Revert the kafka client version upgrade in
> storm-kafka module
>
> Kafka 0.10 brokers have a new message format which can be consumed and
> produced to using older Kafka API, but at the price of performance
>
> Is it posible to build a KafkaStorm support based on Kafka 0.10.x API ?
>
> Best regards,
> Alexandre
>
>
>
>

Mime
View raw message