It should just work in PySpark, the same way it does in Java / Scala apps.


On Oct 1, 2014, at 4:12 PM, Sungwook Yoon <> wrote:

Yes.. you should use maprfs://

I personally haven't used pyspark, I just used scala shell or standalone with MapR.

I think you need to set classpath right, adding jar like
 /opt/mapr/hadoop/hadoop-0.20.2/lib/hadoop-0.20.2-dev-core.jar to the classpath
in the classpath.


On Wed, Oct 1, 2014 at 4:09 PM, Addanki, Santosh Kumar <> wrote:



We would like to do this in PySpark Environment


i.e something like


test = sc.textFile("maprfs:///user/root/test") or

test = sc.textFile("hdfs:///user/root/test") or



Currently when we try

test = sc.textFile("maprfs:///user/root/test")


It throws the error

No File-System for scheme: maprfs



Best Regards





From: Vladimir Rodionov []
Sent: Wednesday, October 01, 2014 3:59 PM
To: Addanki, Santosh Kumar
Subject: Re: Spark And Mapr


There is doc on MapR:


-Vladimir Rodionov


On Wed, Oct 1, 2014 at 3:00 PM, Addanki, Santosh Kumar <> wrote:



We were using Horton 2.4.1 as our Hadoop distribution and now switched to MapR


Previously to read a text file  we would use :


test = sc.textFile(\"hdfs://\")"



What would be the equivalent of the same for Mapr.


Best Regards