spark-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Nicholas Chammas <nicholas.cham...@gmail.com>
Subject Re: Save an RDD to a SQL Database
Date Thu, 07 Aug 2014 15:36:49 GMT
On Thu, Aug 7, 2014 at 11:08 AM, 诺铁 <notyycn@gmail.com> wrote:

> what if network broken in half of the process?  should we drop all data in
> database and restart from beginning?


The best way to deal with this -- which, unfortunately, is not commonly
supported -- is with a two-phase commit that can span connections
<http://stackoverflow.com/q/23354034/877069>. PostgreSQL supports it, for
example.

This would guarantee that a multi-connection data load is atomic.

Nick

Mime
View raw message