spark-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Michael Armbrust <>
Subject Re: how to implement my own datasource?
Date Thu, 25 Jun 2015 18:31:09 GMT
I'd suggest looking at the avro data source as an example implementation:

I also gave a talk a while ago:

You can connect to by JDBC as described in
Other option is using HadoopRDD and NewHadoopRDD to connect to databases
compatible with Hadoop, like HBase, some examples can be found at chapter 5
of "Learning Spark"
For Spark Streaming see the section "Custom Sources" of

Hope that helps.



2015-06-25 8:25 GMT+02:00 诺铁 <>:

> hi,
> I can't find documentation about datasource api,  how to implement custom
> datasource.  any hint is appreciated.    thanks.

View raw message