ignite-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Naden Franciscus (JIRA)" <j...@apache.org>
Subject [jira] [Created] (IGNITE-3078) IgniteContext needs option to read config from HDFS
Date Mon, 02 May 2016 00:31:12 GMT
Naden Franciscus created IGNITE-3078:

             Summary: IgniteContext needs option to read config from HDFS
                 Key: IGNITE-3078
                 URL: https://issues.apache.org/jira/browse/IGNITE-3078
             Project: Ignite
          Issue Type: Bug
            Reporter: Naden Franciscus

If you are running Ignite in YARN mode then obviously we aren't going to have Ignite installation
directories on each of the Hadoop cluster nodes. Therefore we need to store the configuration
in HDFS for it to be accessible everywhere.

What is needed is a new constructor in IgniteContext:

    def this(
        sc: SparkContext,
        hdfsPath: org.apache.hadoop.fs.Path
    ) {
        this(sc, () ⇒ IgnitionEx.loadHDFSConfiguration(hdfsPath).get1())

This message was sent by Atlassian JIRA

View raw message