spark-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Ahmed Nawar <ahmed.na...@gmail.com>
Subject commit DB Transaction for each partition
Date Thu, 27 Aug 2015 14:02:34 GMT
Dears,

    I needs to commit DB Transaction for each partition,Not for each row.
below didn't work for me.


rdd.mapPartitions(partitionOfRecords => {

DBConnectionInit()

val results = partitionOfRecords.map(......)

DBConnection.commit()

results

})



Best regards,

Ahmed Atef Nawwar

Data Management & Big Data Consultant

Mime
View raw message