spark-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Michael Armbrust <mich...@databricks.com>
Subject Re: How can I read this avro file using spark & scala?
Date Thu, 20 Nov 2014 18:30:29 GMT
One option (starting with Spark 1.2, which is currently in preview) is to
use the Avro library for Spark SQL.  This is very new, but we would love to
get feedback: https://github.com/databricks/spark-avro

On Thu, Nov 20, 2014 at 10:19 AM, al b <beanboy3@googlemail.com> wrote:

> I've read several posts of people struggling to read avro in spark. The
> examples I've tried don't work. When I try this solution (
> https://stackoverflow.com/questions/23944615/how-can-i-load-avros-in-spark-using-the-schema-on-board-the-avro-files)
> I get errors:
>
> spark java.io.NotSerializableException: org.apache.avro.mapred.AvroWrapper
>
> How can I read the following sample file in spark using scala?
>
> http://www.4shared.com/file/SxnYcdgJce/sample.html
>
> Thomas
>

Mime
View raw message