spark-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Vida Ha <>
Subject Re: Save an RDD to a SQL Database
Date Thu, 07 Aug 2014 17:52:55 GMT
The use case I was thinking of was outputting calculations made in Spark
into a SQL database for the presentation layer to access.  So in other
words, having a Spark backend in Java that writes to a SQL database and
then having a Rails front-end that can display the data nicely.

On Thu, Aug 7, 2014 at 8:42 AM, Nicholas Chammas <
> wrote:

> On Thu, Aug 7, 2014 at 11:25 AM, Cheng Lian <> wrote:
>> Maybe a little off topic, but would you mind to share your motivation of
>> saving the RDD into an SQL DB?
> Many possible reasons (Vida, please chime in with yours!):
>    - You have an existing database you want to load new data into so
>    everything's together.
>    - You want very low query latency, which you can probably get with
>    Spark SQL but currently not with the ease you can get it from your average
>    DBMS.
>    - Tooling around traditional DBMSs is currently much more mature than
>    tooling around Spark SQL, especially in the JDBC area.
> Nick

View raw message