spark-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Andre Kuhnen <andrekuh...@gmail.com>
Subject Re: How to read a multipart s3 file?
Date Wed, 07 May 2014 00:39:10 GMT
Try using s3n instead of s3
Em 06/05/2014 21:19, "kamatsuoka" <kenjim@gmail.com> escreveu:

> I have a Spark app that writes out a file, s3://mybucket/mydir/myfile.txt.
>
> Behind the scenes, the S3 driver creates a bunch of files like
> s3://mybucket//mydir/myfile.txt/part-0000, as well as the block files like
> s3://mybucket/block_3574186879395643429.
>
> How do I construct an url to use this file as input to another Spark app?
>  I
> tried all the variations of s3://mybucket/mydir/myfile.txt, but none of
> them
> work.
>
>
>
>
>
> --
> View this message in context:
> http://apache-spark-user-list.1001560.n3.nabble.com/How-to-read-a-multipart-s3-file-tp5463.html
> Sent from the Apache Spark User List mailing list archive at Nabble.com.
>

Mime
View raw message