spark-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Marius Soutier <>
Subject SparkSQL and columnar data
Date Thu, 23 Oct 2014 09:47:00 GMT
Hi guys,

another question: what’s the approach to working with column-oriented data, i.e. data with
more than 1000 columns. Using Parquet for this should be fine, but how well does SparkSQL
handle the big amount of columns? Is there a limit? Should we use standard Spark instead?

Thanks for any insights,
- Marius

To unsubscribe, e-mail:
For additional commands, e-mail:

View raw message