spark-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Ryan Blue <>
Subject Re: Behavior of SaveMode.Append when table is not present
Date Fri, 09 Nov 2018 17:13:58 GMT
Right now, it is up to the source implementation to decide what to do. I
think path-based tables (with no metastore component) treat an append as an
implicit create.

If you're thinking that relying on sources to interpret SaveMode is bad for
consistent behavior, I agree. That's why the community adopted a proposal
to standardize logical plans and the behavior
expected of data sources for the v2 API.

On Thu, Nov 8, 2018 at 11:53 PM Shubham Chaurasia <>

> Hi,
> For SaveMode.Append, the doc
> says
> *When saving a DataFrame to a data source, if data/table already exists,
> contents of the DataFrame are expected to be appended to existing data*
> However it does not specify behavior when the table does not exist.
> Does that throw exception or create the table or a NO-OP?
> Thanks,
> Shubham

Ryan Blue
Software Engineer

View raw message