spark-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From spark receiver <spark.recei...@gmail.com>
Subject Re: writing into oracle database is very slow
Date Fri, 19 Apr 2019 08:46:50 GMT
hi Jiang,

i was facing the very same issue ,the solution is write to file and using
oracle external table to do the insert.

hope this could help.

Dalin

On Thu, Apr 18, 2019 at 11:43 AM Jörn Franke <jornfranke@gmail.com> wrote:

> What is the size of the data? How much time does it need on HDFS and how
> much on Oracle? How many partitions do you have on Oracle side?
>
> Am 06.04.2019 um 16:59 schrieb Lian Jiang <jiangok2006@gmail.com>:
>
> Hi,
>
> My spark job writes into oracle db using:
>
> df.coalesce(10).write.format("jdbc").option("url", url)
>   .option("driver", driver).option("user", user)
>   .option("batchsize", 2000)
>   .option("password", password).option("dbtable", tableName).mode("append").save()
>
> It is much slow than writting into HDFS. The data to write is small.
>
> Is this expected? Thanks for any clue.
>
>
>

Mime
View raw message