spark-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Christopher Matta <cma...@mapr.com>
Subject Creating Dataframe from XML parsed by scalaxb
Date Tue, 08 Sep 2015 17:17:18 GMT
I’m attempting to ingest some XML data created by Microsoft Biztalk using
spark. In order to validate and simplify the parsing of the XML I’d like to
use scalaxb <http://scalaxb.org>, however when trying to convert the RDD of
scalaxb objects to a Dataframe, I get the following error:

java.lang.UnsupportedOperationException: Schema for type
scalaxb.DataRecord[scala.Any] is not supported

I’ve written up a more detailed description over on Stack Overflow:
http://stackoverflow.com/questions/32361226/creating-dataframe-from-xml-parsed-by-scalaxb

Is there a way to get the toDF() function to support custom Any types? If
not, is there a better way to go about ingesting SOAP-like XML with Spark
and Scala? I’m new to scalaxb and Scala in general.

Thanks in advance.

Chris Mattacmatta@mapr.com
215-701-3146
​

Mime
View raw message