spark-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Dylan Guedes <djmggue...@gmail.com>
Subject Re: Benchmark Java/Scala/Python for Apache spark
Date Mon, 11 Mar 2019 14:13:43 GMT
Btw, even if you are using Python you can register your UDFs in Scala and
use them in Python.

On Mon, Mar 11, 2019 at 6:55 AM Jonathan Winandy <jonathan.winandy@gmail.com>
wrote:

> Hello Snehasish
>
> If you are not using UDFs, you will have very similar performance with
> those languages on SQL.
>
> So it go down to :
> * if you know python, go for python.
> * if you are used to the JVM, and are ready for a bit of paradigm shift,
> go for Scala.
>
> Our team is using Scala, however we help other data engs that are using
> python.
>
> I would say go for pure functional programming, however that is biased and
> python gets the job done anyway.
>
> Cheers,
> Jonathan
>
> On Mon, 11 Mar 2019, 10:34 SNEHASISH DUTTA, <info.snehasish@gmail.com>
> wrote:
>
>> Hi
>>
>> Is there a way to get performance benchmarks for development of
>> application using either Java/Scala/Python
>>
>> Use case mostly involve SQL pipeline/data ingested from various sources
>> including Kafka
>>
>> What should be the most preferred language and it would be great if the
>> preference for language can be justified from the perspective of
>> application development
>>
>> Thanks and Regards
>> Snehasish
>>
>

Mime
View raw message