spark-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Amit Joshi <mailtojoshia...@gmail.com>
Subject Re: [Spark Kafka Structured Streaming] Adding partition and topic to the kafka dynamically
Date Fri, 28 Aug 2020 04:56:17 GMT
Any pointers will be appreciated.

On Thursday, August 27, 2020, Amit Joshi <mailtojoshiamit@gmail.com> wrote:

> Hi All,
>
> I am trying to understand the effect of adding topics and partitions to a
> topic in kafka, which is being consumed by spark structured streaming
> applications.
>
> Do we have to restart the spark structured streaming application to read
> from the newly added topic?
> Do we have to restart the spark structured streaming application to read
> from the newly added partition to a topic?
>
> Kafka consumers have a meta data refresh property that works without
> restarting.
>
> Thanks advance.
>
> Regards
> Amit Joshi
>

Mime
View raw message