spark-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Sun, Rui" <>
Subject RE: SparkR package path
Date Tue, 22 Sep 2015 03:15:29 GMT

Any strong reason to download and install SparkR source package separately from the Spark
An R user can simply download the spark distribution, which contains SparkR source and binary
package, and directly use sparkR. No need to install SparkR package at all.

From: Hossein []
Sent: Tuesday, September 22, 2015 9:19 AM
Subject: SparkR package path

Hi dev list,

SparkR backend assumes SparkR source files are located under "SPARK_HOME/R/lib/." This directory
is created by running R/ This setting makes sense for Spark developers, but
if an R user downloads and installs SparkR source package, the source files are going to be
in placed different locations.

In the R runtime it is easy to find location of package files using path.package("SparkR").
But we need to make some changes to R backend and/or spark-submit so that, JVM process learns
the location of worker.R and daemon.R and shell.R from the R runtime.

Do you think this change is feasible?

View raw message