spark-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Amiya Mishra <Amiya.Mis...@bitwiseglobal.com>
Subject Re: How to branch a Stream / have multiple Sinks / do multiple Queries on one Stream
Date Fri, 06 Jul 2018 09:12:14 GMT
Hi Tathagata,

Is there any limitation of below code while writing to multiple file ?

val inputdf:DataFrame =
sparkSession.readStream.schema(schema).format("csv").option("delimiter",",").csv("src/main/streamingInput")
    query1 =
inputdf.writeStream.option("path","first_output").option("checkpointLocation","checkpointloc").format("csv").start()
    query2 =
inputdf.writeStream.option("path","second_output").option("checkpointLocation","checkpoint2").format("csv").start()
    sparkSession.streams.awaitAnyTermination()


And what will be the release date of spark 2.4.0 ?

Thanks
Amiya







--
Sent from: http://apache-spark-user-list.1001560.n3.nabble.com/

---------------------------------------------------------------------
To unsubscribe e-mail: user-unsubscribe@spark.apache.org


Mime
View raw message