spark-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Renato Marroquín Mogrovejo <renatoj.marroq...@gmail.com>
Subject Re: Reading parquet files into Spark Streaming
Date Fri, 26 Aug 2016 21:56:14 GMT
Anybody? I think Rory also didn't get an answer from the list ...

https://mail-archives.apache.org/mod_mbox/spark-user/201602.mbox/%3CCAC+fRE14PV5nvQHTBVqDC+6DkXo73oDAzfqsLbSo8F94ozO5nQ@mail.gmail.com%3E



2016-08-26 17:42 GMT+02:00 Renato Marroquín Mogrovejo <
renatoj.marroquin@gmail.com>:

> Hi all,
>
> I am trying to use parquet files as input for DStream operations, but I
> can't find any documentation or example. The only thing I found was [1] but
> I also get the same error as in the post (Class
> parquet.avro.AvroReadSupport not found).
> Ideally I would like to do have something like this:
>
> val oDStream = ssc.fileStream[Void, Order, ParquetInputFormat[Order]]("
> data/")
>
> where Order is a case class and the files inside "data" are all parquet
> files.
> Any hints would be highly appreciated. Thanks!
>
>
> Best,
>
> Renato M.
>
> [1] http://stackoverflow.com/questions/35413552/how-do-i-
> read-in-parquet-files-using-ssc-filestream-and-what-is-the-nature
>

Mime
View raw message