spark-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Nicholas Chammas <nicholas.cham...@gmail.com>
Subject Re: SocketException when reading from S3 (s3n format)
Date Wed, 04 Jun 2014 14:34:51 GMT
I think by default a thread can die up to 4 times before Spark considers it
a failure. Are you seeing that happen? I believe that is a configurable
thing, but don't know off the top of my head how to change it.

I've seen this error before when reading data from a large amount of files
on S3, and it is typically harmless. Spark just retries the operation and
proceeds normally.


On Wed, Jun 4, 2014 at 4:05 AM, yuzeh <delta1080@gmail.com> wrote:

> I should add that I'm using spark 0.9.1.
>
> Thanks!
>
>
>
> --
> View this message in context:
> http://apache-spark-user-list.1001560.n3.nabble.com/SocketException-when-reading-from-S3-s3n-format-tp6889p6890.html
> Sent from the Apache Spark User List mailing list archive at Nabble.com.
>

Mime
View raw message