spark-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From satyajit vegesna <satyajit.apas...@gmail.com>
Subject Json Parsing.
Date Wed, 06 Dec 2017 23:39:05 GMT
Does spark support automatic detection of schema from a json string in a
dataframe.

I am trying to parse a json string and do some transofrmations on to it
(would like append new columns to the dataframe) , from the data i stream
from kafka.

But i am not very sure, how i can parse the json in structured streaming.
And i would not be interested in creating a schema, as the data form kafka
is going to maintain different schema objects in value column.

Any advice or help would be appreciated.

Regards,
Satyajit.

Mime
View raw message