spark-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Charles Feduke <>
Subject Re: Writing Spark Streaming Programs
Date Thu, 19 Mar 2015 15:10:44 GMT
Scala is the language used to write Spark so there's never a situation in
which features introduced in a newer version of Spark cannot be taken
advantage of if you write your code in Scala. (This is mostly true of Java,
but it may be a little more legwork if a Java-friendly adapter isn't
available alongside new features.)

Scala is also OO; its a functional hybrid OO language.

Although much of my organization's codebase is written in Java and we've
recently transitioned to Java 8 I still write all of my Spark code using
Scala. (I also squeeze in Scala where I can in other parts of the
organization.) Additionally I use both Python and R for local data
analysis, though I haven't used Python with Spark in production.

On Thu, Mar 19, 2015 at 10:51 AM James King <> wrote:

> Hello All,
> I'm using Spark for streaming but I'm unclear one which implementation
> language to use Java, Scala or Python.
> I don't know anything about Python, familiar with Scala and have been
> doing Java for a long time.
> I think the above shouldn't influence my decision on which language to use
> because I believe the tool should, fit the problem.
> In terms of performance Java and Scala are comparable. However Java is OO
> and Scala is FP, no idea what Python is.
> If using Scala and not applying a consistent style of programming Scala
> code can become unreadable, but I do like the fact it seems to be possible
> to do so much work with so much less code, that's a strong selling point
> for me. Also it could be that the type of programming done in Spark is best
> implemented in Scala as FP language, not sure though.
> The question I would like your good help with is are there any other
> considerations I need to think about when deciding this? are there any
> recommendations you can make in regards to this?
> Regards
> jk

View raw message