spark-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Doaa Medhat <doaa.m...@gmail.com>
Subject Why "spark-streaming-kafka-0-10" is still experimental?
Date Thu, 04 Apr 2019 07:52:01 GMT
Dears,

I'm working on a project that should integrate spark streaming with kafka
using java.
Currently the official documentation is confusing, it's not clear whether
"spark-streaming-kafka-0-10" is safe to be used in production environment
or not.

According to "Spark Streaming + Kafka Integration Guide"
https://spark.apache.org/docs/latest/streaming-kafka-integration.html
=> "spark-streaming-kafka-0-10" is marked as "Stable" and
"spark-streaming-kafka-0-8" is marked as "Deprecated"

However, in "Spark Streaming + Kafka Integration Guide (Kafka broker
version 0.10.0 or higher)"
https://spark.apache.org/docs/latest/streaming-kafka-0-10-integration.html
=> It's mentioned that "This version of the integration is marked as
experimental, so the API is potentially subject to change." and also all
APIs have the annotation @Experimental. Also, it has been initially
released from around 3 years!!

So, is it safe to use "spark-streaming-kafka-0-10" in production? and why
this version is still marked as experimental till now? and when it's
supposed to be marked as a stable version?

Thanks,

Mime
View raw message