spark-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Ulanov, Alexander" <>
Subject Access to hdfs FileSystem through Spark
Date Fri, 10 Apr 2015 00:23:15 GMT

Is there a way to access hdfs FileSystem through Spark? For example, I need to check the file
size before opening it with sc.binaryFile("hdfs://"). Can I do it
without creating hadoop FileSystem by myself ?

val fs = FileSystem.get(new URI("hdfs://"), new Configuration())

Best regards, Alexander

  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message