spark-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Cody Koeninger <c...@koeninger.org>
Subject Re: Parquet schema migrations
Date Wed, 08 Oct 2014 21:08:14 GMT
On Wed, Oct 8, 2014 at 3:19 PM, Michael Armbrust <michael@databricks.com>
wrote:

>
> I was proposing you manually convert each different format into one
> unified format  (by adding literal nulls and such for missing columns) and
> then union these converted datasets.  It would be weird to have union all
> try and do this automatically.
>


Sure, I was just musing on what an api for doing the merging without manual
user input should look like / do.   I'll comment on the ticket, thanks for
making it

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message