storm-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Aaron Niskodé-Dossett <doss...@gmail.com>
Subject Re: Is it possible to choose Kafa version with StormKafka from Storm 1.0.2 ?
Date Thu, 12 Jan 2017 03:30:36 GMT
Hi Alexandre,

My recollection from trying this when 0.10 was released is that it is
possible, but some small code changes are necessary.  If you're comfortable
building Storm from source, updating the kafka version in the appropriate
poms and compiling would show you what needs to be updated.

On Wed, Jan 11, 2017 at 2:20 PM Alexandre Vermeerbergen <
avermeerbergen@gmail.com> wrote:

> Hello,
>
> Any clue on whether or not it's possible to build a KafkaSpout / KafkaBolt
> based on Apache Kafka 0.10.x ?
>
> Best regards,
> Alexandre
>
>
> 2017-01-07 18:03 GMT+01:00 Alexandre Vermeerbergen <
> avermeerbergen@gmail.com>:
>
> Hello,
>
> We have been happyly running our real-time supervision homebrewed
> application with Apache Storm 1.0.1 and its out-of-the-box StormKafka spout
> & bold with our Kafka Broker 0.9.0.1 cluster for more than 6 months.
>
> We're been recently interested in upgrading our setup to Apache Storm
> 1.0.2, mainly because we sometime hit
> https://issues.apache.org/jira/browse/STORM-1879
>
> We also plan to soon upgrade our Kafka Broker cluster to 0.10.x
>
> However, we noticed that Storm 1.0.2's KafkaSpout has been downgraded to
> Kafka 0.8.2.2 : STORM-1755: Revert the kafka client version upgrade in
> storm-kafka module
>
> Kafka 0.10 brokers have a new message format which can be consumed and
> produced to using older Kafka API, but at the price of performance
>
> Is it posible to build a KafkaStorm support based on Kafka 0.10.x API ?
>
> Best regards,
> Alexandre
>
>
>

Mime
View raw message