spark-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Steve Loughran <ste...@hortonworks.com>
Subject Re: Trying to fetch S3 data
Date Wed, 28 Sep 2016 13:55:13 GMT

On 28 Sep 2016, at 06:28, Hitesh Goyal <hitesh.goyal@nlpcaptcha.com<mailto:hitesh.goyal@nlpcaptcha.com>>
wrote:

Hi team,

I want to fetch data from Amazon S3 bucket. For this, I am trying to access it using scala.
I have tried the basic wordcount application in scala.
Now I want to retrieve s3 data using it.
I have gone through the tutorials and I found solutions for uploading files to S3.
Please tell me how can I retrieve the data buckets stored in S3.


This is actually something I'm trying to document as part of some work to add a spark-cloud
module which adds the appropriate JARs to the classpath for things to work out the box.

Could you have a look at

https://github.com/steveloughran/spark/blob/b04c037f2925d9b698e541493fc936627ddcf9ba/docs/cloud-integration.md

And tell me where it could be improved?


Mime
View raw message