spark-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Toby Douglass <t...@avocet.io>
Subject Re: initial basic question from new user
Date Thu, 12 Jun 2014 15:50:49 GMT
On Thu, Jun 12, 2014 at 4:48 PM, Andre Schumacher <
schumach@icsi.berkeley.edu> wrote:

> On 06/12/2014 05:47 PM, Toby Douglass wrote:
>
> > In these future jobs, when I come to load the aggregted RDD, will Spark
> > load and only load the columns being accessed by the query?  or will
> Spark
> > load everything, to convert it into an internal representation, and then
> > execute the query?
>
> The aforementioned native Parquet support in Spark 1.0 supports column
> projections which means only the columns that appear in the query will
> be loaded.
>

[snip]

Thankyou!

Mime
View raw message