spark-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Felix Cheung <felixcheun...@hotmail.com>
Subject Re: How to propagate R_LIBS to sparkr executors
Date Fri, 18 Nov 2016 02:15:24 GMT
Have you tried
spark.executorEnv.R_LIBS?

spark.apache.org/docs/latest/configuration.html#runtime-environment

_____________________________
From: Rodrick Brown <rodrick@orchard-app.com<mailto:rodrick@orchard-app.com>>
Sent: Wednesday, November 16, 2016 1:01 PM
Subject: How to propagate R_LIBS to sparkr executors
To: <user@spark.apache.org<mailto:user@spark.apache.org>>


I'm having an issue with a R module not getting picked up on the slave nodes in mesos. I have
the following environment value R_LIBS set and for some reason this environment is only set
in the driver context and not the executor is their a way to pass environment values down
the executor level in sparkr?

I'm using Mesos 1.0.1 and Spark 2.0.1

Thanks.


--

[Orchard Platform]<http://www.orchardplatform.com/>

Rodrick Brown / Site Reliability Engineer
+1 917 445 6839 / rodrick@orchardplatform.com<mailto:charlie@orchardplatform.com>

Orchard Platform
101 5th Avenue, 4th Floor, New York, NY 10003
http://www.orchardplatform.com<http://www.orchardplatform.com/>

Orchard Blog<http://www.orchardplatform.com/blog/> | Marketplace Lending Meetup<http://www.meetup.com/Peer-to-Peer-Lending-P2P/>


NOTICE TO RECIPIENTS: This communication is confidential and intended for the use of the addressee
only. If you are not an intended recipient of this communication, please delete it immediately
and notify the sender by return email. Unauthorized reading, dissemination, distribution or
copying of this communication is prohibited. This communication does not constitute an offer
to sell or a solicitation of an indication of interest to purchase any loan, security or any
other financial product or instrument, nor is it an offer to sell or a solicitation of an
indication of interest to purchase any products or services to any persons who are prohibited
from receiving such information under applicable law. The contents of this communication may
not be accurate or complete and are subject to change without notice. As such, Orchard App,
Inc. (including its subsidiaries and affiliates, "Orchard") makes no representation regarding
the accuracy or completeness of the information contained herein. The intended recipient is
advised to consult its own professional advisors, including those specializing in legal, tax
and accounting matters. Orchard does not provide legal, tax or accounting advice.



Mime
View raw message