spark-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Hyukjin Kwon <gurwls...@gmail.com>
Subject Re: How to read a Multi Line json object via Spark
Date Tue, 15 Nov 2016 08:11:02 GMT
Hi Sree,


There is a blog about that,
http://searchdatascience.com/spark-adventures-1-processing-multi-line-json-files/

It is pretty old but I am sure that it is helpful.

Currently, JSON datasource only supports to rest JSON documents formatted
according to http://jsonlines.org/

There is an issue open to support this
https://issues.apache.org/jira/browse/SPARK-18352

I hope this is helpful.


Thanks.



2016-11-15 16:20 GMT+09:00 Sree Eedupuganti <sree@inndata.in>:

> I tried from Spark-Shell and i am getting the following error:
>
> Here is the test.json file:
>
> {
>     "colorsArray": [{
>         "red": "#f00",
>         "green": "#0f0",
>         "blue": "#00f",
>         "cyan": "#0ff",
>         "magenta": "#f0f",
>         "yellow": "#ff0",
>         "black": "#000"
>     }]}
>
>
> scala> val jtex = sqlContext.read.format("json").option("samplingRatio","1.0").load("/user/spark/test.json")
>
>        jtex: org.apache.spark.sql.DataFrame = [_corrupt_record: string]
>
>
> Any suggestions please. Thanks.
> --
> Best Regards,
> Sreeharsha Eedupuganti
> Data Engineer
> innData Analytics Private Limited
>

Mime
View raw message