spark-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Cody Koeninger <>
Subject Re: Parquet schema migrations
Date Wed, 08 Oct 2014 21:08:14 GMT
On Wed, Oct 8, 2014 at 3:19 PM, Michael Armbrust <>

> I was proposing you manually convert each different format into one
> unified format  (by adding literal nulls and such for missing columns) and
> then union these converted datasets.  It would be weird to have union all
> try and do this automatically.

Sure, I was just musing on what an api for doing the merging without manual
user input should look like / do.   I'll comment on the ticket, thanks for
making it

  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message