spark-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Ulanov, Alexander" <alexander.ula...@hp.com>
Subject Access to hdfs FileSystem through Spark
Date Fri, 10 Apr 2015 00:23:15 GMT
Hi,

Is there a way to access hdfs FileSystem through Spark? For example, I need to check the file
size before opening it with sc.binaryFile("hdfs://mynetwork.com:9000/myfile"). Can I do it
without creating hadoop FileSystem by myself ?

val fs = FileSystem.get(new URI("hdfs://mynetwork.com:9000"), new Configuration())

Best regards, Alexander

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message