spark-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Olivier Girardot <ssab...@gmail.com>
Subject Re: Spark 1.3.1 and Parquet Partitions
Date Thu, 07 May 2015 11:06:31 GMT
"hdfs://some ip:8029/dataset/*/*.parquet" doesn't work for you ?

Le jeu. 7 mai 2015 à 03:32, vasuki <vaxuki@gmail.com> a écrit :

> Spark 1.3.1 -
> i have a parquet file on hdfs partitioned by some string looking like this
> /dataset/city=London/data.parquet
> /dataset/city=NewYork/data.parquet
> /dataset/city=Paris/data.paruqet
> ….
>
> I am trying to get to load it using sqlContext using
> sqlcontext.parquetFile(
> "hdfs://some ip:8029/dataset/< what do i put here >
>
> No leads so far. is there i can load the partitions ? I am running on
> cluster and not local..
> -V
>
>
>
> --
> View this message in context:
> http://apache-spark-user-list.1001560.n3.nabble.com/Spark-1-3-1-and-Parquet-Partitions-tp22792.html
> Sent from the Apache Spark User List mailing list archive at Nabble.com.
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: user-unsubscribe@spark.apache.org
> For additional commands, e-mail: user-help@spark.apache.org
>
>

Mime
View raw message