spark-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Ron Gonzalez <zlgonza...@yahoo.com.INVALID>
Subject Re: Save an RDD to a SQL Database
Date Wed, 06 Aug 2014 14:18:06 GMT
Hi Vida,
  It's possible to save an RDD as a hadoop file using hadoop output formats. It might be worthwhile
to investigate using DBOutputFormat and see if this will work for you.
  I haven't personally written to a db, but I'd imagine this would be one way to do it.

Thanks,
Ron

Sent from my iPhone

> On Aug 5, 2014, at 8:29 PM, Vida Ha <vidaha@gmail.com> wrote:
> 
> 
> Hi,
> 
> I would like to save an RDD to a SQL database.  It seems like this would be a common
enough use case.  Are there any built in libraries to do it?
> 
> Otherwise, I'm just planning on mapping my RDD, and having that call a method to write
to the database.   Given that a lot of records are going to be written, the code would need
to be smart and do a batch insert after enough records have collected.  Does that sound like
a reasonable approach?
> 
> 
> -Vida
> 

---------------------------------------------------------------------
To unsubscribe, e-mail: user-unsubscribe@spark.apache.org
For additional commands, e-mail: user-help@spark.apache.org


Mime
View raw message