spark-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Sheel Pancholi <sheelst...@gmail.com>
Subject Are Spark Dataframes mutable in Structured Streaming?
Date Wed, 15 May 2019 18:25:06 GMT
Hi
Structured Streaming treats a stream as an unbounded table in the form of a
DataFrame. Continuously flowing data from the stream keeps getting added to
this DataFrame (which is the unbounded table) which warrants a change to
the DataFrame which violates the vary basic nature of a DataFrame since a
DataFrame by its nature is immutable. This sounds contradictory. Is there
an explanation for this?

Regards
Sheel

Mime
View raw message