spark-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Ofir Manor (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (SPARK-18057) Update structured streaming kafka from 10.0.1 to 10.2.0
Date Sun, 26 Feb 2017 09:21:21 GMT

    [ https://issues.apache.org/jira/browse/SPARK-18057?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15884619#comment-15884619
] 

Ofir Manor commented on SPARK-18057:
------------------------------------

[~cody@koeninger.org] I think my previous issue with upgrading to Kafka 0.10.1 client has
been resolved in Kafka 0.10.2 - since now newer clients DO support older brokers:
{quote}
The Java clients (producer and consumer) have acquired the ability to communicate with older
brokers. Version 0.10.2 clients can talk to version 0.10.0 or newer brokers. Note that some
features are not available or are limited when older brokers are used.
{quote}
See https://kafka.apache.org/documentation/#upgrade_1020_notable
So it seems safe to upgrade Spark's Kafka client to 0.10.2, as it shouldn't bump up the minimum
required version of the Kafka broker.
What do you think?

> Update structured streaming kafka from 10.0.1 to 10.2.0
> -------------------------------------------------------
>
>                 Key: SPARK-18057
>                 URL: https://issues.apache.org/jira/browse/SPARK-18057
>             Project: Spark
>          Issue Type: Improvement
>          Components: Structured Streaming
>            Reporter: Cody Koeninger
>
> There are a couple of relevant KIPs here, https://archive.apache.org/dist/kafka/0.10.1.0/RELEASE_NOTES.html



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@spark.apache.org
For additional commands, e-mail: issues-help@spark.apache.org


Mime
View raw message