Hi,
I am getting error on below scenario. Please suggest.
i have a virtual view in hive
view name log_data
it has 2 columns
query_map map<string,string>
parti_date int
Here is my snippet for the spark data frame
my dataframe
res=sqlcont.sql("select parti_date FROM log_data WHERE parti_date >=
408910 limit 10")
df=res.collect()
print 'after collect'
print df
* File "/usr/local/spark/python/lib/pyspark.zip/pyspark/sql/utils.py", line
51, in deco*
*pyspark.sql.utils.AnalysisException: u'undefined function
json_array_to_map; line 28 pos 73'*
|