spark-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Sheel Pancholi <>
Subject Are Spark Dataframes mutable in Structured Streaming?
Date Wed, 15 May 2019 18:25:06 GMT
Structured Streaming treats a stream as an unbounded table in the form of a
DataFrame. Continuously flowing data from the stream keeps getting added to
this DataFrame (which is the unbounded table) which warrants a change to
the DataFrame which violates the vary basic nature of a DataFrame since a
DataFrame by its nature is immutable. This sounds contradictory. Is there
an explanation for this?


View raw message