spark-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Ron Gonzalez <>
Subject Re: Save an RDD to a SQL Database
Date Wed, 06 Aug 2014 14:18:06 GMT
Hi Vida,
  It's possible to save an RDD as a hadoop file using hadoop output formats. It might be worthwhile
to investigate using DBOutputFormat and see if this will work for you.
  I haven't personally written to a db, but I'd imagine this would be one way to do it.


Sent from my iPhone

> On Aug 5, 2014, at 8:29 PM, Vida Ha <> wrote:
> Hi,
> I would like to save an RDD to a SQL database.  It seems like this would be a common
enough use case.  Are there any built in libraries to do it?
> Otherwise, I'm just planning on mapping my RDD, and having that call a method to write
to the database.   Given that a lot of records are going to be written, the code would need
to be smart and do a batch insert after enough records have collected.  Does that sound like
a reasonable approach?
> -Vida

To unsubscribe, e-mail:
For additional commands, e-mail:

View raw message