spark-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Cody Koeninger <c...@koeninger.org>
Subject Re: KafkaUtils.createStream(..) is removed for API
Date Mon, 19 Feb 2018 21:09:11 GMT
I can't speak for committers, but my guess is it's more likely for
DStreams in general to stop being supported before that particular
integration is removed.

On Sun, Feb 18, 2018 at 9:34 PM, naresh Goud <nareshgoud.dulam@gmail.com> wrote:
> Thanks Ted.
>
> I see  createDirectStream is experimental as annotated with
> "org.apache.spark.annotation.Experimental".
>
> Is it possible to be this API will be removed in future?  because we wanted
> to use this API in one of our production jobs. afraid if it will not be
> supported in future.
>
> Thank you,
> Naresh
>
>
>
>
> On Sun, Feb 18, 2018 at 7:47 PM, Ted Yu <yuzhihong@gmail.com> wrote:
>>
>> createStream() is still in
>> external/kafka-0-8/src/main/scala/org/apache/spark/streaming/kafka/KafkaUtils.scala
>> But it is not in
>> external/kafka-0-10/src/main/scala/org/apache/spark/streaming/kafka010/KafkaUtils.scala
>>
>> FYI
>>
>> On Sun, Feb 18, 2018 at 5:17 PM, naresh Goud <nareshgoud.dulam@gmail.com>
>> wrote:
>>>
>>> Hello Team,
>>>
>>> I see "KafkaUtils.createStream() " method not available in spark 2.2.1.
>>>
>>> Can someone please confirm if these methods are removed?
>>>
>>> below is my pom.xml entries.
>>>
>>>
>>> <properties>
>>>   <scala.version>2.11.8</scala.version>
>>>   <scala.tools.version>2.11</scala.tools.version>
>>> </properties>
>>>
>>>
>>>   <dependency>
>>>       <groupId>org.apache.spark</groupId>
>>>       <artifactId>spark-streaming_${scala.tools.version}</artifactId>
>>>       <version>2.2.1</version>
>>>       <scope>provided</scope>
>>>   </dependency>
>>> <dependency>
>>>   <groupId>org.apache.spark</groupId>
>>>   <artifactId>spark-streaming-kafka-0-10_2.11</artifactId>
>>>   <version>2.2.1</version>
>>>   <scope>provided</scope>
>>> </dependency>
>>> <dependency>
>>>   <groupId>org.apache.spark</groupId>
>>>   <artifactId>spark-core_2.11</artifactId>
>>>   <version>2.2.1</version>
>>>   <scope>provided</scope>
>>> </dependency>
>>>   <dependency>
>>>
>>>
>>>
>>>
>>>
>>> Thank you,
>>> Naresh
>>
>>
>

---------------------------------------------------------------------
To unsubscribe e-mail: user-unsubscribe@spark.apache.org


Mime
View raw message