spark-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Guangle Fan <fanguan...@gmail.com>
Subject Spark Streaming
Date Thu, 17 Jul 2014 23:41:24 GMT
Hi, All

When I run spark streaming, in one of the flatMap stage, I want to access
database.

Code looks like :

stream.flatMap(
new FlatMapFunction {
    call () {
        //access database cluster
    }
  }
)

Since I don't want to create database connection every time call() was
called, where is the best place do I create the connection and reuse it on
per-host basis (Like one database connection per Mapper/Reducer ) ?

Regards,

Guangle

Mime
View raw message