spark-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Matei Zaharia <>
Subject Re: Is Spark in Java a bad idea?
Date Tue, 28 Oct 2014 18:56:39 GMT
A pretty large fraction of users use Java, but a few features are still not available in it.
JdbcRDD is one of them -- this functionality will likely be superseded by Spark SQL when we
add JDBC as a data source. In the meantime, to use it, I'd recommend writing a class in Scala
that has Java-friendly methods and getting an RDD to it from that. Basically the two parameters
that weren't friendly there were the ClassTag and the getConnection and mapRow functions.

Subclassing RDD in Java is also not really supported, because that's an internal API. We don't
expect users to be defining their own RDDs.


> On Oct 28, 2014, at 11:47 AM, critikaled <> wrote:
> Hi Ron,
> what ever api you have in scala you can possibly use it form java. scala is
> inter-operable with java and vice versa. scala being both object oriented
> and functional will make your job easier on jvm and it is more consise than
> java. Take it as an opportunity and start learning scala ;).
> --
> View this message in context:
> Sent from the Apache Spark User List mailing list archive at
> ---------------------------------------------------------------------
> To unsubscribe, e-mail:
> For additional commands, e-mail:

To unsubscribe, e-mail:
For additional commands, e-mail:

View raw message