spark-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "vdiwakar.malladi" <>
Subject Re: loading, querying schemaRDD using SparkSQL
Date Thu, 13 Nov 2014 12:35:11 GMT
Thanks Michael.

I used Parquet files and it could able to solve my initial problem to some
extent (i.e. loading data from one context and reading it from another

But there I could see another issue. I need to load the parquet file every
time I create the JavaSQLContext using parquetFile method on that (for the
creation of JavaSchemaRDD) and need to register as temp table using
registerTempTable for the querying purpose. It seems to be a problem when
our web application is in cluster mode. Because I need to load the parquet
files on each node. Could you please advice me on this?

Thanks in advance.

View this message in context:
Sent from the Apache Spark User List mailing list archive at

To unsubscribe, e-mail:
For additional commands, e-mail:

View raw message