spark-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From kamatsuoka <>
Subject How to read a multipart s3 file?
Date Wed, 07 May 2014 00:19:07 GMT
I have a Spark app that writes out a file, s3://mybucket/mydir/myfile.txt.

Behind the scenes, the S3 driver creates a bunch of files like
s3://mybucket//mydir/myfile.txt/part-0000, as well as the block files like

How do I construct an url to use this file as input to another Spark app?  I
tried all the variations of s3://mybucket/mydir/myfile.txt, but none of them

View this message in context:
Sent from the Apache Spark User List mailing list archive at

View raw message