phoenix-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Chinmay Kulkarni (JIRA)" <j...@apache.org>
Subject [jira] [Created] (PHOENIX-5197) Use extraOptions to set the configuration for Spark Workers
Date Thu, 14 Mar 2019 21:07:00 GMT
Chinmay Kulkarni created PHOENIX-5197:
-----------------------------------------

             Summary: Use extraOptions to set the configuration for Spark Workers
                 Key: PHOENIX-5197
                 URL: https://issues.apache.org/jira/browse/PHOENIX-5197
             Project: Phoenix
          Issue Type: Improvement
    Affects Versions: 5.0.0, 4.15.0
            Reporter: Chinmay Kulkarni
            Assignee: Chinmay Kulkarni


We need a standardized way to set configurations for Spark driver and workers when reading
and writing data.

Quoting from my offline discussion with [~twdsilva@gmail.com]:

"_if we use phoenixTableAsRDD to create the rdd then we can pass in a config
if we use the standard df.write.format() we can't pass in a conf
maybe the best was to do this is to put config in the option map when we call read/write .format
and then set those in the config _"



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

Mime
View raw message